OpenRaiser/Intern-Atlas huggingface dataset:
Fast BATLLNN: Fast Box Analysis of Two-Level Lattice Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

In this paper, we present the tool Fast Box Analysis of Two-Level Lattice Neural Networks (Fast BATLLNN) as a fast verifier of box-like output constraints for Two-Level Lattice (TLL) Neural Networks (NNs). In particular, Fast BATLLNN can verify whether the output of a given TLL NN always lies within a specified hyper-rectangle whenever its input constrained to a specified convex polytope (not necessarily a hyper-rectangle). Fast BATLLNN uses the unique semantics of the TLL architecture and the decoupled nature of box-like output constraints to dramatically improve verification performance relative to known polynomial-time verification algorithms for TLLs with generic polytopic output constraints. In this paper, we evaluate the performance and scalability of Fast BATLLNN, both in its own right and compared to state-of-the-art NN verifiers applied to TLL NNs. Fast BATLLNN compares very favorably to even the fastest NN verifiers, completing our synthetic TLL test bench more than 400x faster than its nearest competitor.
An adapted deflated conjugate gradient solver for robust extended/generalised finite element solutions of large scale, 3D crack propagation problems
Venue: arXiv cs.CE 2021
Year: 2021

An adapted deflation preconditioner is employed to accelerate the solution of linear systems resulting from the discretization of fracture mechanics problems with well-conditioned extended/generalized finite elements. The deflation space typically used for linear elasticity problems is enriched with additional vectors, accounting for the enrichment functions used, thus effectively removing low frequency components of the error. To further improve performance, deflation is combined, in a multiplicative way, with a block-Jacobi preconditioner, which removes high frequency components of the error as well as linear dependencies introduced by enrichment. The resulting scheme is tested on a series of non-planar crack propagation problems and compared to alternative linear solvers in terms of performance.
Stream Sampling with Immediate Decision
Venue: arXiv cs.DS 2021
Year: 2021

The manuscript introduces a method to select a random sample from a stream by deciding on each sampling unit immediately after observing it. The process could be applied to unequal as well as equal probability sampling. The implementation is straightforward. Algorithm selects a unit in the sample based on a single condition. It is particularly effective to make direct decisions on stream data, despite the data arriving in groups or the stream being linear.
Case study of SARS-CoV-2 transmission risk assessment in indoor environments using cloud computing resources
Venue: arXiv cs.DC 2021
Year: 2021

Complex flow simulations are conventionally performed on HPC clusters. However, the limited availability of HPC resources and steep learning curve of executing on traditional supercomputer infrastructure has drawn attention towards deploying flow simulation software on the cloud. We showcase how a complex computational framework -- that can evaluate COVID-19 transmission risk in various indoor classroom scenarios -- can be abstracted and deployed on cloud services. The availability of such cloud-based personalized planning tools can enable educational institutions, medical institutions, public sector workers (courthouses, police stations, airports, etc.), and other entities to comprehensively evaluate various in-person interaction scenarios for transmission risk. We deploy the simulation framework on the Azure cloud framework, utilizing the Dendro-kT mesh generation tool and PETSc solvers. The cloud abstraction is provided by RocketML cloud infrastructure. We compare the performance of the cloud machines with state-of-the-art HPC machine TACC Frontera. Our results suggest that cloud-based HPC resources are a viable strategy for a diverse array of end-users to rapidly and efficiently deploy simulation software.
Submodular Optimization for Coupled Task Allocation and Intermittent Deployment Problems
Venue: arXiv cs.RO 2021
Year: 2021

In this paper, we demonstrate a formulation for optimizing coupled submodular maximization problems with provable sub-optimality bounds. In robotics applications, it is quite common that optimization problems are coupled with one another and therefore cannot be solved independently. Specifically, we consider two problems coupled if the outcome of the first problem affects the solution of a second problem that operates over a longer time scale. For example, in our motivating problem of environmental monitoring, we posit that multi-robot task allocation will potentially impact environmental dynamics and thus influence the quality of future monitoring, here modeled as a multi-robot intermittent deployment problem. The general theoretical approach for solving this type of coupled problem is demonstrated through this motivating example. Specifically, we propose a method for solving coupled problems modeled by submodular set functions with matroid constraints. A greedy algorithm for solving this class of problem is presented, along with sub-optimality guarantees. Finally, practical optimality ratios are shown through Monte Carlo simulations to demonstrate that the proposed algorithm can generate near-optimal solutions with high efficiency.
Highly Accurate and Reliable Wireless Network Slicing in 5th Generation Networks: A Hybrid Deep Learning Approach
Venue: arXiv cs.NI 2021
Year: 2021

In the current era, the next-generation networks like 5th generation (5G) and 6th generation (6G) networks require high security, low latency with a high reliable standards and capacity. In these networks, reconfigurable wireless network slicing is considered as one of the key elements for 5G and 6G networks. A reconfigurable slicing allows the operators to run various instances of the network using a single infrastructure for a better quality of services (QoS). The QoS can be achieved by reconfiguring and optimizing these networks using Artificial intelligence and machine learning algorithms. To develop a smart decision-making mechanism for network management and restricting network slice failures, machine learning-enabled reconfigurable wireless network solutions are required. In this paper, we propose a hybrid deep learning model that consists of a convolution neural network (CNN) and long short term memory (LSTM). The CNN performs resource allocation, network reconfiguration, and slice selection while the LSTM is used for statistical information (load balancing, error rate etc.) regarding network slices. The applicability of the proposed model is validated by using multiple unknown devices, slice failure, and overloading conditions. The overall accuracy of 95.17% is achieved by the proposed model that reflects its applicability.
Blind Calibration of Air Quality Wireless Sensor Networks Using Deep Neural Networks
Venue: arXiv cs.NI 2021
Year: 2021

Temporal drift of low-cost sensors is crucial for the applicability of wireless sensor networks (WSN) to measure highly local phenomenon such as air quality. The emergence of wireless sensor networks in locations without available reference data makes calibrating such networks without the aid of true values a key area of research. While deep learning (DL) has proved successful on numerous other tasks, it is under-researched in the context of blind WSN calibration, particularly in scenarios with networks that mix static and mobile sensors. In this paper we investigate the use of DL architectures for such scenarios, including the effects of weather in both drifting and sensor measurement. New models are proposed and compared against a baseline, based on a previous proposed model and extended to include mobile sensors and weather data. Also, a procedure for generating simulated air quality data is presented, including the emission, dispersion and measurement of the two most common particulate matter pollutants: PM 2.5 and PM 10 . Results show that our models reduce the calibration error with an order of magnitude compared to the baseline, showing that DL is a suitable method for WSN calibration and that these networks can be remotely calibrated with minimal cost for the deployer.
On the Effectiveness of Iterative Learning Control
Venue: arXiv cs.RO 2021
Year: 2021

Iterative learning control (ILC) is a powerful technique for high performance tracking in the presence of modeling errors for optimal control applications. There is extensive prior work showing its empirical effectiveness in applications such as chemical reactors, industrial robots and quadcopters. However, there is little prior theoretical work that explains the effectiveness of ILC even in the presence of large modeling errors, where optimal control methods using the misspecified model (MM) often perform poorly. Our work presents such a theoretical study of the performance of both ILC and MM on Linear Quadratic Regulator (LQR) problems with unknown transition dynamics. We show that the suboptimality gap, as measured with respect to the optimal LQR controller, for ILC is lower than that for MM by higher order terms that become significant in the regime of high modeling errors. A key part of our analysis is the perturbation bounds for the discrete Ricatti equation in the finite horizon setting, where the solution is not a fixed point and requires tracking the error using recursive bounds. We back our theoretical findings with empirical experiments on a toy linear dynamical system with an approximate model, a nonlinear inverted pendulum system with misspecified mass, and a nonlinear planar quadrotor system in the presence of wind. Experiments show that ILC outperforms MM significantly, in terms of the cost of computed trajectories, when modeling errors are high.
Self-Attending Task Generative Adversarial Network for Realistic Satellite Image Creation
Venue: arXiv cs.LG 2021
Year: 2021

We introduce a self-attending task generative adversarial network (SATGAN) and apply it to the problem of augmenting synthetic high contrast scientific imagery of resident space objects with realistic noise patterns and sensor characteristics learned from collected data. Augmenting these synthetic data is challenging due to the highly localized nature of semantic content in the data that must be preserved. Real collected images are used to train a network what a given class of sensor's images should look like. The trained network then acts as a filter on noiseless context images and outputs realistic-looking fakes with semantic content unaltered. The architecture is inspired by conditional GANs but is modified to include a task network that preserves semantic information through augmentation. Additionally, the architecture is shown to reduce instances of hallucinatory objects or obfuscation of semantic content in context images representing space observation scenes.
Software Engineering for Responsible AI: An Empirical Study and Operationalised Patterns
Venue: arXiv cs.AI 2021
Year: 2021

Although artificial intelligence (AI) is solving real-world challenges and transforming industries, there are serious concerns about its ability to behave and make decisions in a responsible way. Many AI ethics principles and guidelines for responsible AI have been recently issued by governments, organisations, and enterprises. However, these AI ethics principles and guidelines are typically high-level and do not provide concrete guidance on how to design and develop responsible AI systems. To address this shortcoming, we first present an empirical study where we interviewed 21 scientists and engineers to understand the practitioners' perceptions on AI ethics principles and their implementation. We then propose a template that enables AI ethics principles to be operationalised in the form of concrete patterns and suggest a list of patterns using the newly created template. These patterns provide concrete, operationalised guidance that facilitate the development of responsible AI systems.
A Novel Optimized Asynchronous Federated Learning Framework
Venue: arXiv cs.LG 2021
Year: 2021

Federated Learning (FL) since proposed has been applied in many fields, such as credit assessment, medical, etc. Because of the difference in the network or computing resource, the clients may not update their gradients at the same time that may take a lot of time to wait or idle. That's why Asynchronous Federated Learning (AFL) method is needed. The main bottleneck in AFL is communication. How to find a balance between the model performance and the communication cost is a challenge in AFL. This paper proposed a novel AFL framework VAFL. And we verified the performance of the algorithm through sufficient experiments. The experiments show that VAFL can reduce the communication times about 51.02\% with 48.23\% average communication compression rate and allow the model to be converged faster. The code is available at \url{https://github.com/RobAI-Lab/VAFL}
Blind VQA on 360° Video via Progressively Learning from Pixels, Frames and Video
Venue: arXiv cs.CV 2021
Year: 2021

Blind visual quality assessment (BVQA) on 360{\textdegree} video plays a key role in optimizing immersive multimedia systems. When assessing the quality of 360{\textdegree} video, human tends to perceive its quality degradation from the viewport-based spatial distortion of each spherical frame to motion artifact across adjacent frames, ending with the video-level quality score, i.e., a progressive quality assessment paradigm. However, the existing BVQA approaches for 360{\textdegree} video neglect this paradigm. In this paper, we take into account the progressive paradigm of human perception towards spherical video quality, and thus propose a novel BVQA approach (namely ProVQA) for 360{\textdegree} video via progressively learning from pixels, frames and video. Corresponding to the progressive learning of pixels, frames and video, three sub-nets are designed in our ProVQA approach, i.e., the spherical perception aware quality prediction (SPAQ), motion perception aware quality prediction (MPAQ) and multi-frame temporal non-local (MFTN) sub-nets. The SPAQ sub-net first models the spatial quality degradation based on spherical perception mechanism of human. Then, by exploiting motion cues across adjacent frames, the MPAQ sub-net properly incorporates motion contextual information for quality assessment on 360{\textdegree} video. Finally, the MFTN sub-net aggregates multi-frame quality degradation to yield the final quality score, via exploring long-term quality correlation from multiple frames. The experiments validate that our approach significantly advances the state-of-the-art BVQA performance on 360{\textdegree} video over two datasets, the code of which has been public in \url{https://github.com/yanglixiaoshen/ProVQA.}
On Clustering with Discounts
Venue: arXiv cs.DS 2021
Year: 2021

We study the $k$-median with discounts problem, wherein we are given clients with non-negative discounts and seek to open at most $k$ facilities. The goal is to minimize the sum of distances from each client to its nearest open facility which is discounted by its own discount value, with minimum contribution being zero. $k$-median with discounts unifies many classic clustering problems, e.g., $k$-center, $k$-median, $k$-facility $l$-centrum, etc. We obtain a bi-criteria constant-factor approximation using an iterative LP rounding algorithm. Our result improves the previously best approximation guarantee for $k$-median with discounts [Ganesh et al., ICALP'21]. We also devise bi-criteria constant-factor approximation algorithms for the matroid and knapsack versions of median clustering with discounts.
How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN
Venue: arXiv cs.CL 2021
Year: 2021

Current language models can generate high-quality text. Are they simply copying text they have seen before, or have they learned generalizable linguistic abstractions? To tease apart these possibilities, we introduce RAVEN, a suite of analyses for assessing the novelty of generated text, focusing on sequential structure (n-grams) and syntactic structure. We apply these analyses to four neural language models (an LSTM, a Transformer, Transformer-XL, and GPT-2). For local structure - e.g., individual dependencies - model-generated text is substantially less novel than our baseline of human-generated text from each model's test set. For larger-scale structure - e.g., overall sentence structure - model-generated text is as novel or even more novel than the human-generated baseline, but models still sometimes copy substantially, in some cases duplicating passages over 1,000 words long from the training set. We also perform extensive manual analysis showing that GPT-2's novel text is usually well-formed morphologically and syntactically but has reasonably frequent semantic issues (e.g., being self-contradictory).
How Emotionally Stable is ALBERT? Testing Robustness with Stochastic Weight Averaging on a Sentiment Analysis Task
Venue: arXiv cs.CL 2021
Year: 2021

Despite their success, modern language models are fragile. Even small changes in their training pipeline can lead to unexpected results. We study this phenomenon by examining the robustness of ALBERT (arXiv:1909.11942) in combination with Stochastic Weight Averaging (SWA) (arXiv:1803.05407) -- a cheap way of ensembling -- on a sentiment analysis task (SST-2). In particular, we analyze SWA's stability via CheckList criteria (arXiv:2005.04118), examining the agreement on errors made by models differing only in their random seed. We hypothesize that SWA is more stable because it ensembles model snapshots taken along the gradient descent trajectory. We quantify stability by comparing the models' mistakes with Fleiss' Kappa (Fleiss, 1971) and overlap ratio scores. We find that SWA reduces error rates in general; yet the models still suffer from their own distinct biases (according to CheckList).
Evaluating Transformers for Lightweight Action Recognition
Venue: arXiv cs.CV 2021
Year: 2021

In video action recognition, transformers consistently reach state-of-the-art accuracy. However, many models are too heavyweight for the average researcher with limited hardware resources. In this work, we explore the limitations of video transformers for lightweight action recognition. We benchmark 13 video transformers and baselines across 3 large-scale datasets and 10 hardware devices. Our study is the first to evaluate the efficiency of action recognition models in depth across multiple devices and train a wide range of video transformers under the same conditions. We categorize current methods into three classes and show that composite transformers that augment convolutional backbones are best at lightweight action recognition, despite lacking accuracy. Meanwhile, attention-only models need more motion modeling capabilities and stand-alone attention block models currently incur too much latency overhead. Our experiments conclude that current video transformers are not yet capable of lightweight action recognition on par with traditional convolutional baselines, and that the previously mentioned shortcomings need to be addressed to bridge this gap. Code to reproduce our experiments will be made publicly available.
CLMB: deep contrastive learning for robust metagenomic binning
Venue: arXiv cs.LG 2021
Year: 2021

The reconstruction of microbial genomes from large metagenomic datasets is a critical procedure for finding uncultivated microbial populations and defining their microbial functional roles. To achieve that, we need to perform metagenomic binning, clustering the assembled contigs into draft genomes. Despite the existing computational tools, most of them neglect one important property of the metagenomic data, that is, the noise. To further improve the metagenomic binning step and reconstruct better metagenomes, we propose a deep Contrastive Learning framework for Metagenome Binning (CLMB), which can efficiently eliminate the disturbance of noise and produce more stable and robust results. Essentially, instead of denoising the data explicitly, we add simulated noise to the training data and force the deep learning model to produce similar and stable representations for both the noise-free data and the distorted data. Consequently, the trained model will be robust to noise and handle it implicitly during usage. CLMB outperforms the previous state-of-the-art binning methods significantly, recovering the most near-complete genomes on almost all the benchmarking datasets (up to 17\% more reconstructed genomes compared to the second-best method). It also improves the performance of bin refinement, reconstructing 8-22 more high-quality genomes and 15-32 more middle-quality genomes than the second-best result. Impressively, in addition to being compatible with the binning refiner, single CLMB even recovers on average 15 more HQ genomes than the refiner of VAMB and Maxbin on the benchmarking datasets. CLMB is open-source and available at https://github.com/zpf0117b/CLMB/.
A Low-Cost, Easy-to-Manufacture, Flexible, Multi-Taxel Tactile Sensor and its Application to In-Hand Object Recognition
Venue: arXiv cs.RO 2021
Year: 2021

Soft robotics is an emerging field that yields promising results for tasks that require safe and robust interactions with the environment or with humans, such as grasping, manipulation, and human-robot interaction. Soft robots rely on intrinsically compliant components and are difficult to equip with traditional, rigid sensors which would interfere with their compliance. We propose a highly flexible tactile sensor that is low-cost and easy to manufacture while measuring contact pressures independently from 14 taxels. The sensor is built from piezoresistive fabric for highly sensitive, continuous responses and from a custom-designed flexible printed circuit board which provides a high taxel density. From these taxels, location and intensity of contact with the sensor can be inferred. In this paper, we explain the design and manufacturing of the proposed sensor, characterize its input-output relation, evaluate its effects on compliance when equipped to the silicone-based pneumatic actuators of the soft robotic RBO Hand 2, and demonstrate that the sensor provides rich and useful feedback for learning-based in-hand object recognition.
Measuring source code conciseness across programming languages using compression
Venue: arXiv cs.PL 2021
Year: 2021

It is well-known, and often a topic of heated debates, that programs in some programming languages are more concise than in others. This is a relevant factor when comparing or aggregating volume-impacted metrics on source code written in a combination of programming languages. In this paper, we present a model for measuring the conciseness of programming languages in a consistent, objective and evidence-based way. We present the approach, explain how it is founded on information theoretical principles, present detailed analysis steps and show the quantitative results of applying this model to a large benchmark of diverse commercial software applications. We demonstrate that our metric for language conciseness is strongly correlated with both an alternative analytical approach, and with a large scale developer survey, and show how its results can be applied to improve software metrics for multi-language applications.