OpenRaiser/Intern-Atlas huggingface dataset:
Achievable rate-region for $3-$User Classical-Quantum Interference Channel using Structured Codes
Venue: arXiv cs.IT 2021
Year: 2021

We consider the problem of characterizing an inner bound to the capacity region of a $3-$user classical-quantum interference channel ($3-$CQIC). The best known coding scheme for communicating over CQICs is based on unstructured random codes and employs the techniques of message splitting and superposition coding. For classical $3-$user interference channels (ICs), it has been proven that coding techniques based on coset codes - codes possessing algebraic closure properties - strictly outperform all coding techniques based on unstructured codes. In this work, we develop analogous techniques based on coset codes for $3$to$1-$CQICs - a subclass of $3-$user CQICs. We analyze its performance and derive a new inner bound to the capacity region of $3$to$1-$CQICs that subsume the current known largest and strictly enlarges the same for identified examples.
Over-the-Air Statistical Estimation
Venue: arXiv cs.IT 2021
Year: 2021

We study schemes and lower bounds for distributed minimax statistical estimation over a Gaussian multiple-access channel (MAC) under squared error loss, in a framework combining statistical estimation and wireless communication. First, we develop "analog" joint estimation-communication schemes that exploit the superposition property of the Gaussian MAC and we characterize their risk in terms of the number of nodes and dimension of the parameter space. Then, we derive information-theoretic lower bounds on the minimax risk of any estimation scheme restricted to communicate the samples over a given number of uses of the channel and show that the risk achieved by our proposed schemes is within a logarithmic factor of these lower bounds. We compare both achievability and lower bound results to previous "digital" lower bounds, where nodes transmit errorless bits at the Shannon capacity of the MAC, showing that estimation schemes that leverage the physical layer offer a drastic reduction in estimation error over digital schemes relying on a physical-layer abstraction.
Black-Box IoT: Authentication and Distributed Storage of IoT Data from Constrained Sensors
Venue: arXiv cs.CR 2021
Year: 2021

We propose Black-Box IoT (BBox-IoT), a new ultra-lightweight black-box system for authenticating and storing IoT data. BBox-IoT is tailored for deployment on IoT devices (including low-Size Weight and Power sensors) which are extremely constrained in terms of computation, storage, and power. By utilizing core Blockchain principles, we ensure that the collected data is immutable and tamper-proof while preserving data provenance and non-repudiation. To realize BBox-IoT, we designed and implemented a novel chain-based hash signature scheme which only requires hashing operations and removes all synchronicity dependencies between signer and verifier. Our approach enables low-SWaP devices to authenticate removing reliance on clock synchronization. Our evaluation results show that BBox-IoT is practical in Industrial Internet of Things (IIoT) environments: even devices equipped with 16MHz micro-controllers and 2KB memory can broadcast their collected data without requiring heavy cryptographic operations or synchronicity assumptions. Finally, when compared to industry standard ECDSA, our approach is two and three orders of magnitude faster for signing and verification operations respectively. Thus, we are able to increase the total number of signing operations by more than 5000% for the same amount of power.
Linear Regression over Networks with Communication Guarantees
Venue: arXiv cs.LG 2021
Year: 2021

A key functionality of emerging connected autonomous systems such as smart cities, smart transportation systems, and the industrial Internet-of-Things, is the ability to process and learn from data collected at different physical locations. This is increasingly attracting attention under the terms of distributed learning and federated learning. However, in connected autonomous systems, data transfer takes place over communication networks with often limited resources. This paper examines algorithms for communication-efficient learning for linear regression tasks by exploiting the informativeness of the data. The developed algorithms enable a tradeoff between communication and learning with theoretical performance guarantees and efficient practical implementations.
Changing the Narrative Perspective: From Deictic to Anaphoric Point of View
Venue: arXiv cs.CL 2021
Year: 2021

We introduce the task of changing the narrative point of view, where characters are assigned a narrative perspective that is different from the one originally used by the writer. The resulting shift in the narrative point of view alters the reading experience and can be used as a tool in fiction writing or to generate types of text ranging from educational to self-help and self-diagnosis. We introduce a benchmark dataset containing a wide range of types of narratives annotated with changes in point of view from deictic (first or second person) to anaphoric (third person) and describe a pipeline for processing raw text that relies on a neural architecture for mention selection. Evaluations on the new benchmark dataset show that the proposed architecture substantially outperforms the baselines by generating mentions that are less ambiguous and more natural.
HexDom: Polycube-Based Hexahedral-Dominant Mesh Generation
Venue: arXiv cs.GR 2021
Year: 2021

In this paper, we extend our earlier polycube-based all-hexahedral mesh generation method to hexahedral-dominant mesh generation, and present the HexDom software package. Given the boundary representation of a solid model, HexDom creates a hex-dominant mesh by using a semi-automated polycube-based mesh generation method. The resulting hexahedral dominant mesh includes hexahedra, tetrahedra, and triangular prisms. By adding non-hexahedral elements, we are able to generate better quality hexahedral elements than in all-hexahedral meshes. We explain the underlying algorithms in four modules including segmentation, polycube construction, hex-dominant mesh generation and quality improvement, and use a rockerarm model to explain how to run the software. We also apply our software to a number of other complex models to test their robustness. The software package and all tested models are availabe in github (https://github.com/CMU-CBML/HexDom).
Synthesis with Asymptotic Resource Bounds
Venue: arXiv cs.PL 2021
Year: 2021

We present a method for synthesizing recursive functions that satisfy both a functional specification and an asymptotic resource bound. Prior methods for synthesis with a resource metric require the user to specify a concrete expression exactly describing resource usage, whereas our method uses big-O notation to specify the asymptotic resource usage. Our method can synthesize programs with complex resource bounds, such as a sort function that has complexity O(nlog(n)). Our synthesis procedure uses a type system that is able to assign an asymptotic complexity to terms, and can track recurrence relations of functions. These typing rules are justified by theorems used in analysis of algorithms, such as the Master Theorem and the Akra-Bazzi method. We implemented our method as an extension of prior type-based synthesis work. Our tool, SynPlexity, was able to synthesize complex divide-and-conquer programs that cannot be synthesized by prior solvers.
Bottlenecks in Blockchain Consensus Protocols
Venue: arXiv cs.DC 2021
Year: 2021

Most of the Blockchain permissioned systems employ Byzantine fault-tolerance (BFT) consensus protocols to ensure that honest validators agree on the order for appending entries to their ledgers. In this paper, we study the performance and the scalability of prominent consensus protocols, namely PBFT, Tendermint, HotStuff, and Streamlet, both analytically via load formulas and practically via implementation and evaluation. Under identical conditions, we identify the bottlenecks of these consensus protocols and show that these protocols do not scale well as the number of validators increases. Our investigation points to the communication complexity as the culprit. Even when there is enough network bandwidth, the CPU cost of serialization and deserialization of the messages limits the throughput and increases the latency of the protocols. To alleviate the bottlenecks, the most useful techniques include reducing the communication complexity, rotating the hotspot of communications, and pipelining across consensus instances.
Student-Teacher Feature Pyramid Matching for Anomaly Detection
Venue: arXiv cs.CV 2021
Year: 2021

Anomaly detection is a challenging task and usually formulated as an one-class learning problem for the unexpectedness of anomalies. This paper proposes a simple yet powerful approach to this issue, which is implemented in the student-teacher framework for its advantages but substantially extends it in terms of both accuracy and efficiency. Given a strong model pre-trained on image classification as the teacher, we distill the knowledge into a single student network with the identical architecture to learn the distribution of anomaly-free images and this one-step transfer preserves the crucial clues as much as possible. Moreover, we integrate the multi-scale feature matching strategy into the framework, and this hierarchical feature matching enables the student network to receive a mixture of multi-level knowledge from the feature pyramid under better supervision, thus allowing to detect anomalies of various sizes. The difference between feature pyramids generated by the two networks serves as a scoring function indicating the probability of anomaly occurring. Due to such operations, our approach achieves accurate and fast pixel-level anomaly detection. Very competitive results are delivered on the MVTec anomaly detection dataset, superior to the state of the art ones.
Tendon-Driven Soft Robotic Gripper for Berry Harvesting
Venue: arXiv cs.RO 2021
Year: 2021

Global berry production and consumption have significantly increased in recent years, coinciding with increased consumer awareness of the health-promoting benefits of berries. Among them, fresh market blackberries and raspberries are primarily harvested by hand to maintain post-harvest quality. However, fresh market berry harvesting is an arduous, costly endeavor that accounts for up to 50% of the worker hours. Additionally, the inconsistent forces applied during hand-harvesting can result in an 85% loss of marketable berries due to red drupelet reversion (RDR). Herein, we present a novel, tendon-driven soft robotic gripper with active contact force feedback control, which leverages the passive compliance of the gripper for the gentle harvesting of blackberries. The versatile gripper was able to apply a desired force as low as 0.5 N with a mean error of 0.046 N, while also holding payloads that produce forces as high as 18 N. Field test results indicate that the gripper is capable of harvesting berries with minimal berry damage, while maintaining a harvesting reliability of 95% and a harvesting rate of approximately 4.8 seconds per berry.
RFN-Nest: An end-to-end residual fusion network for infrared and visible images
Venue: arXiv cs.CV 2021
Year: 2021

In the image fusion field, the design of deep learning-based fusion methods is far from routine. It is invariably fusion-task specific and requires a careful consideration. The most difficult part of the design is to choose an appropriate strategy to generate the fused image for a specific task in hand. Thus, devising learnable fusion strategy is a very challenging problem in the community of image fusion. To address this problem, a novel end-to-end fusion network architecture (RFN-Nest) is developed for infrared and visible image fusion. We propose a residual fusion network (RFN) which is based on a residual architecture to replace the traditional fusion approach. A novel detail-preserving loss function, and a feature enhancing loss function are proposed to train RFN. The fusion model learning is accomplished by a novel two-stage training strategy. In the first stage, we train an auto-encoder based on an innovative nest connection (Nest) concept. Next, the RFN is trained using the proposed loss functions. The experimental results on public domain data sets show that, compared with the existing methods, our end-to-end fusion network delivers a better performance than the state-of-the-art methods in both subjective and objective evaluation. The code of our fusion method is available at https://github.com/hli1221/imagefusion-rfn-nest
Joint Coding and Scheduling Optimization for Distributed Learning over Wireless Edge Networks
Venue: arXiv cs.NI 2021
Year: 2021

Unlike theoretical distributed learning (DL), DL over wireless edge networks faces the inherent dynamics/uncertainty of wireless connections and edge nodes, making DL less efficient or even inapplicable under the highly dynamic wireless edge networks (e.g., using mmW interfaces). This article addresses these problems by leveraging recent advances in coded computing and the deep dueling neural network architecture. By introducing coded structures/redundancy, a distributed learning task can be completed without waiting for straggling nodes. Unlike conventional coded computing that only optimizes the code structure, coded distributed learning over the wireless edge also requires to optimize the selection/scheduling of wireless edge nodes with heterogeneous connections, computing capability, and straggling effects. However, even neglecting the aforementioned dynamics/uncertainty, the resulting joint optimization of coding and scheduling to minimize the distributed learning time turns out to be NP-hard. To tackle this and to account for the dynamics and uncertainty of wireless connections and edge nodes, we reformulate the problem as a Markov Decision Process and then design a novel deep reinforcement learning algorithm that employs the deep dueling neural network architecture to find the jointly optimal coding scheme and the best set of edge nodes for different learning tasks without explicit information about the wireless environment and edge nodes' straggling parameters. Simulations show that the proposed framework reduces the average learning delay in wireless edge computing up to 66% compared with other DL approaches. The jointly optimal framework in this article is also applicable to any distributed learning scheme with heterogeneous and uncertain computing nodes.
Graph Force Learning
Venue: arXiv cs.LG 2021
Year: 2021

Features representation leverages the great power in network analysis tasks. However, most features are discrete which poses tremendous challenges to effective use. Recently, increasing attention has been paid on network feature learning, which could map discrete features to continued space. Unfortunately, current studies fail to fully preserve the structural information in the feature space due to random negative sampling strategy during training. To tackle this problem, we study the problem of feature learning and novelty propose a force-based graph learning model named GForce inspired by the spring-electrical model. GForce assumes that nodes are in attractive forces and repulsive forces, thus leading to the same representation with the original structural information in feature learning. Comprehensive experiments on benchmark datasets demonstrate the effectiveness of the proposed framework. Furthermore, GForce opens up opportunities to use physics models to model node interaction for graph learning.
Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees
Venue: arXiv cs.CL 2021
Year: 2021

Pre-trained language models like BERT achieve superior performances in various NLP tasks without explicit consideration of syntactic information. Meanwhile, syntactic information has been proved to be crucial for the success of NLP applications. However, how to incorporate the syntax trees effectively and efficiently into pre-trained Transformers is still unsettled. In this paper, we address this problem by proposing a novel framework named Syntax-BERT. This framework works in a plug-and-play mode and is applicable to an arbitrary pre-trained checkpoint based on Transformer architecture. Experiments on various datasets of natural language understanding verify the effectiveness of syntax trees and achieve consistent improvement over multiple pre-trained models, including BERT, RoBERTa, and T5.
Improving Text-to-SQL with Schema Dependency Learning
Venue: arXiv cs.CL 2021
Year: 2021

Text-to-SQL aims to map natural language questions to SQL queries. The sketch-based method combined with execution-guided (EG) decoding strategy has shown a strong performance on the WikiSQL benchmark. However, execution-guided decoding relies on database execution, which significantly slows down the inference process and is hence unsatisfactory for many real-world applications. In this paper, we present the Schema Dependency guided multi-task Text-to-SQL model (SDSQL) to guide the network to effectively capture the interactions between questions and schemas. The proposed model outperforms all existing methods in both the settings with or without EG. We show the schema dependency learning partially cover the benefit from EG and alleviates the need for it. SDSQL without EG significantly reduces time consumption during inference, sacrificing only a small amount of performance and provides more flexibility for downstream applications.
Teaching Model-based Requirements Engineering to Industry Professionals: An Experience Report
Venue: arXiv cs.SE 2021
Year: 2021

The use of conceptual models to foster requirements engineering has been proposed and evaluated as beneficial for several decades. For instance, goal-oriented requirements engineering or the specification of scenarios are commonly done using conceptual models. Bringing such model-based requirements engineering approaches into industrial practice typically requires industrial training. In this paper, we report lessons learned from a training program for teaching industry professionals model-based requirements engineering. Particularly, we as educators and learners report experiences from designing the training program, conducting the actual training, and applying the instructed material in our day-to-day work. From these findings we provide guidelines for educators designing requirements engineering courses for industry professionals.
Efficient Model Performance Estimation via Feature Histories
Venue: arXiv cs.LG 2021
Year: 2021

An important step in the task of neural network design, such as hyper-parameter optimization (HPO) or neural architecture search (NAS), is the evaluation of a candidate model's performance. Given fixed computational resources, one can either invest more time training each model to obtain more accurate estimates of final performance, or spend more time exploring a greater variety of models in the configuration space. In this work, we aim to optimize this exploration-exploitation trade-off in the context of HPO and NAS for image classification by accurately approximating a model's maximal performance early in the training process. In contrast to recent accelerated NAS methods customized for certain search spaces, e.g., requiring the search space to be differentiable, our method is flexible and imposes almost no constraints on the search space. Our method uses the evolution history of features of a network during the early stages of training to build a proxy classifier that matches the peak performance of the network under consideration. We show that our method can be combined with multiple search algorithms to find better solutions to a wide range of tasks in HPO and NAS. Using a sampling-based search algorithm and parallel computing, our method can find an architecture which is better than DARTS and with an 80% reduction in wall-clock search time.
Comparative Analysis and Enhancement of CFG-based Hardware-Assisted CFI Schemes
Venue: arXiv cs.AR 2021
Year: 2021

Subverting the flow of instructions (e.g., by use of code-reuse attacks) still poses a serious threat to the security of today's systems. Various control flow integrity (CFI) schemes have been proposed as a powerful technique to detect and mitigate such attacks. In recent years, many hardware-assisted implementations of CFI enforcement based on control flow graphs (CFGs) have been presented by academia. Such approaches check whether control flow transfers follow the intended CFG by limiting the valid target addresses. However, these papers all target different platforms and were evaluated with different sets of benchmark applications, which makes quantitative comparisons hardly possible. For this paper, we have implemented multiple promising CFG-based CFI schemes on a common platform comprising a RISC-V SoC within an FPGA. By porting almost 40 benchmark applications to this system we can present a meaningful comparison of the various techniques in terms of run-time performance, hardware utilization, and binary size. In addition, we present an enhanced CFI approach that is inspired by what we consider the best concepts and ideas of previously proposed mechanisms. We have made this approach more practical and feature-complete by tackling some problems largely ignored previously. We show with this fine-grained scheme that CFI can be achieved with even less overheads than previously demonstrated.
Rapidly-exploring Random Forest: Adaptively Exploits Local Structure with Generalised Multi-Trees Motion Planning
Venue: arXiv cs.RO 2021
Year: 2021

Sampling-based motion planners perform exceptionally well in robotic applications that operate in high-dimensional space. However, most works often constrain the planning workspace rooted at some fixed locations, do not adaptively reason on strategy in narrow passages, and ignore valuable local structure information. In this paper, we propose Rapidly-exploring Random Forest (RRF*) -- a generalised multi-trees motion planner that combines the rapid exploring property of tree-based methods and adaptively learns to deploys a Bayesian local sampling strategy in regions that are deemed to be bottlenecks. Local sampling exploits the local-connectivity of spaces via Markov Chain random sampling, which is updated sequentially with a Bayesian proposal distribution to learns the local structure from past observations. The trees selection problem is formulated as a multi-armed bandit problem, which efficiently allocates resources on the most promising tree to accelerate planning runtime. RRF* learns the region that is difficult to perform tree extensions and adaptively deploys local sampling in those regions to maximise the benefit of exploiting local structure. We provide rigorous proofs of completeness and optimal convergence guarantees, and we experimentally demonstrate that the effectiveness of RRF*'s adaptive multi-trees approach allows it to performs well in a wide range of problems.
Localization and Mapping using Instance-specific Mesh Models
Venue: arXiv cs.CV 2021
Year: 2021

This paper focuses on building semantic maps, containing object poses and shapes, using a monocular camera. This is an important problem because robots need rich understanding of geometry and context if they are to shape the future of transportation, construction, and agriculture. Our contribution is an instance-specific mesh model of object shape that can be optimized online based on semantic information extracted from camera images. Multi-view constraints on the object shape are obtained by detecting objects and extracting category-specific keypoints and segmentation masks. We show that the errors between projections of the mesh model and the observed keypoints and masks can be differentiated in order to obtain accurate instance-specific object shapes. We evaluate the performance of the proposed approach in simulation and on the KITTI dataset by building maps of car poses and shapes.