OpenRaiser/Intern-Atlas huggingface dataset:
pPython for Parallel Python Programming
Venue: arXiv cs.DC 2022
Year: 2022

pPython seeks to provide a parallel capability that provides good speed-up without sacrificing the ease of programming in Python by implementing partitioned global array semantics (PGAS) on top of a simple file-based messaging library (PythonMPI) in pure Python. The core data structure in pPython is a distributed numerical array whose distribution onto multiple processors is specified with a map construct. Communication operations between distributed arrays are abstracted away from the user and pPython transparently supports redistribution between any block-cyclic-overlapped distributions in up to four dimensions. pPython follows a SPMD (single program multiple data) model of computation. pPython runs on any combination of heterogeneous systems that support Python, including Windows, Linux, and MacOS operating systems. In addition to running transparently on single-node (e.g., a laptop), pPython provides a scheduler interface, so that pPython can be executed in a massively parallel computing environment. The initial implementation uses the Slurm scheduler. Performance of pPython on the HPC Challenge benchmark suite demonstrates both ease of programming and scalability.
Geometrical tilings : distance, topology, compactness and completeness
Venue: arXiv cs.DM 2022
Year: 2022

We present the different distances on tilings of Rd that exist in the literature, we prove that (most of) these definitions are correct (i.e. they indeed define metrics on tilings of Rd ). We prove that for subshifts with finite local complexity (FLC) these metrics are topologically equivalent and even metrically equivalent, and also we present classical results of compactness and completeness. Note that, excluding the equivalence of these metrics, all of the results presented here are known (see for example the survey [Rob04]) however we were unable to find a reference with complete proofs for some of these results so we decided to write this notice to clarify some definitions and give full proofs.
HyTGraph: GPU-Accelerated Graph Processing with Hybrid Transfer Management
Venue: arXiv cs.DC 2022
Year: 2022

Processing large graphs with memory-limited GPU needs to resolve issues of host-GPU data transfer, which is a key performance bottleneck. Existing GPU-accelerated graph processing frameworks reduce the data transfers by managing the active subgraph transfer at runtime. Some frameworks adopt explicit transfer management approaches based on explicit memory copy with filter or compaction. In contrast, others adopt implicit transfer management approaches based on on-demand access with zero-copy or unified-memory. Having made intensive analysis, we find that as the active vertices evolve, the performance of the two approaches varies in different workloads. Due to heavy redundant data transfers, high CPU compaction overhead, or low bandwidth utilization, adopting a single approach often results in suboptimal performance. In this work, we propose a hybrid transfer management approach to take the merits of both the two approaches at runtime, with an objective to achieve the shortest execution time in each iteration. Based on the hybrid approach, we present HytGraph, a GPU-accelerated graph processing framework, which is empowered by a set of effective task scheduling optimizations to improve the performance. Our experimental results on real-world and synthesized graphs demonstrate that HyTGraph achieves up to 10.27X speedup over existing GPU-accelerated graph processing systems including Grus, Subway, and EMOGI.
Explainable Artificial Intelligence Applications in Cyber Security: State-of-the-Art in Research
Venue: arXiv cs.CR 2022
Year: 2022

This survey presents a comprehensive review of current literature on Explainable Artificial Intelligence (XAI) methods for cyber security applications. Due to the rapid development of Internet-connected systems and Artificial Intelligence in recent years, Artificial Intelligence including Machine Learning (ML) and Deep Learning (DL) has been widely utilized in the fields of cyber security including intrusion detection, malware detection, and spam filtering. However, although Artificial Intelligence-based approaches for the detection and defense of cyber attacks and threats are more advanced and efficient compared to the conventional signature-based and rule-based cyber security strategies, most ML-based techniques and DL-based techniques are deployed in the black-box manner, meaning that security experts and customers are unable to explain how such procedures reach particular conclusions. The deficiencies of transparency and interpretability of existing Artificial Intelligence techniques would decrease human users' confidence in the models utilized for the defense against cyber attacks, especially in current situations where cyber attacks become increasingly diverse and complicated. Therefore, it is essential to apply XAI in the establishment of cyber security models to create more explainable models while maintaining high accuracy and allowing human users to comprehend, trust, and manage the next generation of cyber defense mechanisms. Although there are papers reviewing Artificial Intelligence applications in cyber security areas and the vast literature on applying XAI in many fields including healthcare, financial services, and criminal justice, the surprising fact is that there are currently no survey research articles that concentrate on XAI applications in cyber security.
MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model
Venue: arXiv cs.CV 2022
Year: 2022

Human motion modeling is important for many modern graphics applications, which typically require professional skills. In order to remove the skill barriers for laymen, recent motion generation methods can directly generate human motions conditioned on natural languages. However, it remains challenging to achieve diverse and fine-grained motion generation with various text inputs. To address this problem, we propose MotionDiffuse, the first diffusion model-based text-driven motion generation framework, which demonstrates several desired properties over existing methods. 1) Probabilistic Mapping. Instead of a deterministic language-motion mapping, MotionDiffuse generates motions through a series of denoising steps in which variations are injected. 2) Realistic Synthesis. MotionDiffuse excels at modeling complicated data distribution and generating vivid motion sequences. 3) Multi-Level Manipulation. MotionDiffuse responds to fine-grained instructions on body parts, and arbitrary-length motion synthesis with time-varied text prompts. Our experiments show MotionDiffuse outperforms existing SoTA methods by convincing margins on text-driven motion generation and action-conditioned motion generation. A qualitative analysis further demonstrates MotionDiffuse's controllability for comprehensive motion generation. Homepage: https://mingyuan-zhang.github.io/projects/MotionDiffuse.html
Be Your Own Neighborhood: Detecting Adversarial Example by the Neighborhood Relations Built on Self-Supervised Learning
Venue: arXiv cs.LG 2022
Year: 2022

Deep Neural Networks (DNNs) have achieved excellent performance in various fields. However, DNNs' vulnerability to Adversarial Examples (AE) hinders their deployments to safety-critical applications. This paper presents a novel AE detection framework, named BEYOND, for trustworthy predictions. BEYOND performs the detection by distinguishing the AE's abnormal relation with its augmented versions, i.e. neighbors, from two prospects: representation similarity and label consistency. An off-the-shelf Self-Supervised Learning (SSL) model is used to extract the representation and predict the label for its highly informative representation capacity compared to supervised learning models. For clean samples, their representations and predictions are closely consistent with their neighbors, whereas those of AEs differ greatly. Furthermore, we explain this observation and show that by leveraging this discrepancy BEYOND can effectively detect AEs. We develop a rigorous justification for the effectiveness of BEYOND. Furthermore, as a plug-and-play model, BEYOND can easily cooperate with the Adversarial Trained Classifier (ATC), achieving the state-of-the-art (SOTA) robustness accuracy. Experimental results show that BEYOND outperforms baselines by a large margin, especially under adaptive attacks. Empowered by the robust relation net built on SSL, we found that BEYOND outperforms baselines in terms of both detection ability and speed. Our code will be publicly available.
Class-Aware Attention for Multimodal Trajectory Prediction
Venue: arXiv cs.CV 2022
Year: 2022

Predicting the possible future trajectories of the surrounding dynamic agents is an essential requirement in autonomous driving. These trajectories mainly depend on the surrounding static environment, as well as the past movements of those dynamic agents. Furthermore, the multimodal nature of agent intentions makes the trajectory prediction problem more challenging. All of the existing models consider the target agent as well as the surrounding agents similarly, without considering the variation of physical properties. In this paper, we present a novel deep-learning based framework for multimodal trajectory prediction in autonomous driving, which considers the physical properties of the target and surrounding vehicles such as the object class and their physical dimensions through a weighted attention module, that improves the accuracy of the predictions. Our model has achieved the highest results in the nuScenes trajectory prediction benchmark, out of the models which use rasterized maps to input environment information. Furthermore, our model is able to run in real-time, achieving a high inference rate of over 300 FPS.
Feynman on Artificial Intelligence and Machine Learning, with Updates
Venue: arXiv cs.AI 2022
Year: 2022

I present my recollections of Richard Feynman's mid-1980s interest in artificial intelligence and neural networks, set in the technical context of the physics-related approaches to neural networks of that time. I attempt to evaluate his ideas in the light of the substantial advances in the field since then, and vice versa. There are aspects of Feynman's interests that I think have been largely achieved and others that remain excitingly open, notably in computational science, and potentially including the revival of symbolic methods therein.
RecLight: A Recurrent Neural Network Accelerator with Integrated Silicon Photonics
Venue: arXiv cs.LG 2022
Year: 2022

Recurrent Neural Networks (RNNs) are used in applications that learn dependencies in data sequences, such as speech recognition, human activity recognition, and anomaly detection. In recent years, newer RNN variants, such as GRUs and LSTMs, have been used for implementing these applications. As many of these applications are employed in real-time scenarios, accelerating RNN/LSTM/GRU inference is crucial. In this paper, we propose a novel photonic hardware accelerator called RecLight for accelerating simple RNNs, GRUs, and LSTMs. Simulation results indicate that RecLight achieves 37x lower energy-per-bit and 10% better throughput compared to the state-of-the-art.
In Alexa, We Trust. Or Do We? : An analysis of People's Perception of Privacy Policies
Venue: arXiv cs.HC 2022
Year: 2022

Smart home devices have found their way through people's homes as well as hearts. One such smart device is Amazon Alexa. Amazon Alexa is a voice-controlled application that is rapidly gaining popularity. Alexa was primarily used for checking weather forecasts, playing music, and controlling other devices. This paper tries to explore the extent to which people are aware of the privacy policies pertaining to the Amazon Alexa devices. We have evaluated behavioral change towards their interactions with the device post being aware of the adverse implications. Resulting knowledge will give researchers new avenues of research and interaction designers new insights into improving their systems.
RAPTOR: Ravenous Throughput Computing
Venue: arXiv cs.DC 2022
Year: 2022

We describe the design, implementation and performance of the RADICAL-Pilot task overlay (RAPTOR). RAPTOR enables the execution of heterogeneous tasks -- i.e., functions and executables with arbitrary duration -- on HPC platforms, providing high throughput and high resource utilization. RAPTOR supports the high throughput virtual screening requirements of DOE's National Virtual Biotechnology Laboratory effort to find therapeutic solutions for COVID-19. RAPTOR has been used on $>8000$ compute nodes to sustain 144M/hour docking hits, and to screen $\sim$10$^{11}$ ligands. To the best of our knowledge, both the throughput rate and aggregated number of executed tasks are a factor of two greater than previously reported in literature. RAPTOR represents important progress towards improvement of computational drug discovery, in terms of size of libraries screened, and for the possibility of generating training data fast enough to serve the last generation of docking surrogate models.
LaserSVG: Responsive Laser-Cutter Templates
Venue: arXiv cs.HC 2022
Year: 2022

Laser cutters take vector data for the shapes they cut or engrave as input, however, re-using a given design with different material or on a different machine requires adaptation of the template. Unfortunately, vector drawings lack the semantic information required for an automated adjustment to new parameters, making the manual adjustment a tedious and error-prone process for end-users. We present LaserSVG, a standard-compliant vector-based file format, software library, and authoring tool to specify, generate, exchange and re-use responsive laser-cutting templates. With LaserSVG, designers can easily turn their vector-drawings into parametric templates that end-users can easily adjust to new materials or production parameters. Our tools provide various functions for parametric design that allows end-users and designers to adapt objects while ensuring overall consistency of the results.
Tree-Based Adaptive Model Learning
Venue: arXiv cs.FL 2022
Year: 2022

We extend the Kearns-Vazirani learning algorithm to be able to handle systems that change over time. We present a new learning algorithm that can reuse and update previously learned behavior, implement it in the LearnLib library, and evaluate it on large examples, to which we make small adjustments between two runs of the algorithm. In these experiments our algorithm significantly outperforms both the classic Kearns-Vazirani learning algorithm and the current state-of-the-art adaptive algorithm.
Partial Counterfactual Identification for Infinite Horizon Partially Observable Markov Decision Process
Venue: arXiv cs.LG 2022
Year: 2022

This paper investigates the problem of bounding possible output from a counterfactual query given a set of observational data. While various works of literature have described methodologies to generate efficient algorithms that provide an optimal bound for the counterfactual query, all of them assume a finite-horizon causal diagram. This paper aims to extend the previous work by modifying Q-learning algorithm to provide informative bounds of a causal query given an infinite-horizon causal diagram. Through simulations, our algorithms are proven to perform better compared to existing algorithm.
A Note on the Games-Chan Algorithm
Venue: arXiv cs.SC 2022
Year: 2022

The Games-Chan algorithm finds the minimal period of a periodic binary sequence of period $2^n$, in $n$ iterations. We generalise this to periodic $q$-ary sequences (where $q$ is a prime power) using generating functions and polynomials and apply this to find the multiplicity of $x-1$ in a $q$-ary polynomial $f$ in $\log_{\,q}°(f)$ iterations.
A Modified IEEE 802.15.6 MAC Scheme to Enhance Performance of Wireless Body Area Networks in E-health Applications
Venue: arXiv cs.NI 2022
Year: 2022

The recently released IEEE 802.15.6 standard specifies several physical (PHY) layers and medium access control (MAC) layer protocols for variety of medical and non-medical applications of Wireless Body Area Networks (WBAN). The medical applications of WBAN has several obligatory requirements and constrains viz. high reliability, strict delay deadlines and low power consumption. The standard IEEE 802.15.6 MAC scheme is not able to fulfil the all requirements of medical applications of WBAN. To address this issue we propose an IEEE 802.15.6-based MAC scheme that is the modification of superframe structure, user priorities and access mechanism of standard IEEE 802.15.6 MAC scheme. The proposed superframe has three access phases: random access phases (RAP), manage access phases (MAP) and contention access phase (CAP). The proposed four user priorities nodes access the channel during RAP using CAMA/CA mechanism with a large value of contention window. The proposed MAC scheme uses RTS/CTS access mechanism instead of basic access mechanism to mitigate the effect of hidden and expose terminal problem. Moreover, we develop an analytical model to evaluate the performance of proposed MAC scheme and solve the analytical model using Maple. The results show that the modified IEEE 802.15.6 MAC scheme achieve the better performance in terms of reliability, throughput, average access delay, energy consumption, channel utilization and fairness compared to standard IEEE 802.15.6 MAC scheme in E-health applications.
Exploring Effective Information Utilization in Multi-Turn Topic-Driven Conversations
Venue: arXiv cs.CL 2022
Year: 2022

Conversations are always related to certain topics. However, it is challenging to fuse dialogue history and topic information from various sources at the same time in current dialogue generation models because of the input length limit of pre-trained language models (PLMs). In order to expand the information that PLMs can utilize, we encode topic and dialogue history information using certain prompts with multiple channels of Fusion-in-Decoder (FiD) and explore the influence of three different channel settings. In this paper, our experiments focus on a specific Chinese dataset named NaturalConv, where the conversation revolves around a piece of recent news. We thoroughly compared different dialogue models and different FiD channel settings. Empirical results show that by combining our proposed whole passage channel with additional history channel, our methods can achieve competitive performance on NaturalConv, making it possible to encode various information from excessively long texts.
Which anonymization technique is best for which NLP task? -- It depends. A Systematic Study on Clinical Text Processing
Venue: arXiv cs.CL 2022
Year: 2022

Clinical text processing has gained more and more attention in recent years. The access to sensitive patient data, on the other hand, is still a big challenge, as text cannot be shared without legal hurdles and without removing personal information. There are many techniques to modify or remove patient related information, each with different strengths. This paper investigates the influence of different anonymization techniques on the performance of ML models using multiple datasets corresponding to five different NLP tasks. Several learnings and recommendations are presented. This work confirms that particularly stronger anonymization techniques lead to a significant drop of performance. In addition to that, most of the presented techniques are not secure against a re-identification attack based on similarity search.
Monocular Camera-based Complex Obstacle Avoidance via Efficient Deep Reinforcement Learning
Venue: arXiv cs.RO 2022
Year: 2022

Deep reinforcement learning has achieved great success in laser-based collision avoidance works because the laser can sense accurate depth information without too much redundant data, which can maintain the robustness of the algorithm when it is migrated from the simulation environment to the real world. However, high-cost laser devices are not only difficult to deploy for a large scale of robots but also demonstrate unsatisfactory robustness towards the complex obstacles, including irregular obstacles, e.g., tables, chairs, and shelves, as well as complex ground and special materials. In this paper, we propose a novel monocular camera-based complex obstacle avoidance framework. Particularly, we innovatively transform the captured RGB images to pseudo-laser measurements for efficient deep reinforcement learning. Compared to the traditional laser measurement captured at a certain height that only contains one-dimensional distance information away from the neighboring obstacles, our proposed pseudo-laser measurement fuses the depth and semantic information of the captured RGB image, which makes our method effective for complex obstacles. We also design a feature extraction guidance module to weight the input pseudo-laser measurement, and the agent has more reasonable attention for the current state, which is conducive to improving the accuracy and efficiency of the obstacle avoidance policy.
One Ontology to Rule Them All: Corner Case Scenarios for Autonomous Driving
Venue: arXiv cs.RO 2022
Year: 2022

The core obstacle towards a large-scale deployment of autonomous vehicles currently lies in the long tail of rare events. These are extremely challenging since they do not occur often in the utilized training data for deep neural networks. To tackle this problem, we propose the generation of additional synthetic training data, covering a wide variety of corner case scenarios. As ontologies can represent human expert knowledge while enabling computational processing, we use them to describe scenarios. Our proposed master ontology is capable to model scenarios from all common corner case categories found in the literature. From this one master ontology, arbitrary scenario-describing ontologies can be derived. In an automated fashion, these can be converted into the OpenSCENARIO format and subsequently executed in simulation. This way, also challenging test and evaluation scenarios can be generated.
Versatile Single-Loop Method for Gradient Estimator: First and Second Order Optimality, and its Application to Federated Learning
Venue: arXiv cs.LG 2022
Year: 2022

While variance reduction methods have shown great success in solving large scale optimization problems, many of them suffer from accumulated errors and, therefore, should periodically require the full gradient computation. In this paper, we present a single-loop algorithm named SLEDGE (Single-Loop mEthoD for Gradient Estimator) for finite-sum nonconvex optimization, which does not require periodic refresh of the gradient estimator but achieves nearly optimal gradient complexity. Unlike existing methods, SLEDGE has the advantage of versatility; (i) second-order optimality, (ii) exponential convergence in the PL region, and (iii) smaller complexity under less heterogeneity of data. We build an efficient federated learning algorithm by exploiting these favorable properties. We show the first and second-order optimality of the output and also provide analysis under PL conditions. When the local budget is sufficiently large and clients are less (Hessian-)~heterogeneous, the algorithm requires fewer communication rounds then existing methods such as FedAvg, SCAFFOLD, and Mime. The superiority of our method is verified in numerical experiments.