OpenRaiser/Intern-Atlas huggingface dataset:
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
Venue: arXiv cs.CL 2019
Year: 2019

We present BART, a denoising autoencoder for pretraining sequence-to-sequence models. BART is trained by (1) corrupting text with an arbitrary noising function, and (2) learning a model to reconstruct the original text. It uses a standard Tranformer-based neural machine translation architecture which, despite its simplicity, can be seen as generalizing BERT (due to the bidirectional encoder), GPT (with the left-to-right decoder), and many other more recent pretraining schemes. We evaluate a number of noising approaches, finding the best performance by both randomly shuffling the order of the original sentences and using a novel in-filling scheme, where spans of text are replaced with a single mask token. BART is particularly effective when fine tuned for text generation but also works well for comprehension tasks. It matches the performance of RoBERTa with comparable training resources on GLUE and SQuAD, achieves new state-of-the-art results on a range of abstractive dialogue, question answering, and summarization tasks, with gains of up to 6 ROUGE. BART also provides a 1.1 BLEU increase over a back-translation system for machine translation, with only target language pretraining. We also report ablation experiments that replicate other pretraining schemes within the BART framework, to better measure which factors most influence end-task performance.
Towards Scalable Continuous-Time Trajectory Optimization for Multi-Robot Navigation
Venue: arXiv cs.RO 2019
Year: 2019

Scalable multi-robot transition is essential for ubiquitous adoption of robots. As a step towards it, a computationally efficient decentralized algorithm for continuous-time trajectory optimization in multi-robot scenarios based upon model predictive control is introduced. The robots communicate only their current states and goals rather than sharing their whole trajectory; using this data each robot predicts a continuous-time trajectory for every other robot exploiting optimal control based motion primitives that are corrected for spatial inter-robot interactions using least squares. A non linear program (NLP) is formulated for collision avoidance with the predicted trajectories of other robots. The NLP is condensed by using time as a parametrization resulting in an unconstrained optimization problem and can be solved in a fast and efficient manner. Additionally, the algorithm resizes the robot to accommodate it's trajectory tracking error. The algorithm was tested in simulations on Gazebo with aerial robots. Early results indicate that the proposed algorithm is efficient for upto forty homogeneous robots and twenty one heterogeneous robots occupying 20\% of the available space.
Ordered Memory
Venue: arXiv cs.LG 2019
Year: 2019

Stack-augmented recurrent neural networks (RNNs) have been of interest to the deep learning community for some time. However, the difficulty of training memory models remains a problem obstructing the widespread use of such models. In this paper, we propose the Ordered Memory architecture. Inspired by Ordered Neurons (Shen et al., 2018), we introduce a new attention-based mechanism and use its cumulative probability to control the writing and erasing operation of the memory. We also introduce a new Gated Recursive Cell to compose lower-level representations into higher-level representation. We demonstrate that our model achieves strong performance on the logical inference task (Bowman et al., 2015)and the ListOps (Nangia and Bowman, 2018) task. We can also interpret the model to retrieve the induced tree structure, and find that these induced structures align with the ground truth. Finally, we evaluate our model on the Stanford SentimentTreebank tasks (Socher et al., 2013), and find that it performs comparatively with the state-of-the-art methods in the literature.
A Novel Procedure for the Assessment of the Seismic Performance of Frame Structures by Means of Limit Analysis
Venue: arXiv cs.CE 2019
Year: 2019

Limit analysis is a computationally efficient tool to assess the resistance and the failure mode of structures but does not provide any information on the displacement capacity, which is one of the concepts which most affects the seismic safety. Therefore, since many researchers did not consider limit analysis as a possible tool for the seismic assessment of structures, its widespread employment has been prevented. In this paper this common belief is questioned and the authors show that limit analysis can be useful in the evaluation of the seismic performance of frame structures. In particular, to overcome the limitation on the possibility to evaluate the displacements of a structure based on a limit analysis approach, an approximated capacity curve is reconstructed. The latter is based on a limit analysis strategy, which takes into account the second order effects, and evaluates the displacement capacity considering a post-peak softening branch and a threshold on the allowed plastic rotations. Then, based on this simplified capacity curve, an equivalent single degree of freedom system is defined in order to assess the seismic performance of frame structures. The proposed simplified strategy is implemented in a dedicated software and the obtained results are validated with well-established approaches based on nonlinear static analyses, showing the reliability and the computational efficiency of this methodology
The Tradeoff Between Coverage and Computation in Wireless Networks
Venue: arXiv cs.IT 2019
Year: 2019

We consider a distributed edge computing scenario consisting of several wireless nodes that are located over an area of interest. Specifically, some of the "master" nodes are tasked to sense the environment (e.g., by acquiring images or videos via cameras) and process the corresponding sensory data, while the other nodes are assigned as "workers" to help the computationally-intensive processing tasks of the masters. A new tradeoff that has not been previously explored in the existing literature arises in such a formulation: On one hand, one wishes to allocate as many master nodes as possible to cover a large area for accurate monitoring. On the other hand, one also wishes to allocate as many worker nodes as possible to maximize the computation rate of the sensed data. It is in the context of this tradeoff that this work is presented. By utilizing the basic physical layer principles of wireless communication systems, we formulate and analyze the tradeoff between the coverage and computation performance of spatial networks. We also present an algorithm to find the optimal tradeoff and demonstrate its performance through numerical simulations.
Achieving Optimal Backlog in the Vanilla Multi-Processor Cup Game
Venue: arXiv cs.DS 2019
Year: 2019

In each step of the $p$-processor cup game on $n$ cups, a filler distributes up to $p$ units of water among the cups, subject only to the constraint that no cup receives more than $1$ unit of water; an emptier then removes up to $1$ unit of water from each of $p$ cups. Designing strategies for the emptier that minimize backlog (i.e., the height of the fullest cup) is important for applications in processor scheduling, buffer management in networks, quality of service guarantees, and deamortization. We prove that the greedy algorithm (i.e., the empty-from-fullest-cups algorithm) achieves backlog $O(\log n)$ for any $p \ge 1$. This resolves a long-standing open problem for $p > 1$, and is asymptotically optimal as long as $n \ge 2p$. If the filler is an oblivious adversary, then we prove that there is a randomized emptying algorithm that achieve backlog $O(\log p + \log \log n)$ with probability $1 - 2^{-\operatorname{polylog}(n)}$ for $2^{\operatorname{polylog}(n)}$ steps. This is known to be asymptotically optimal when $n$ is sufficiently large relative to $p$. The analysis of the randomized algorithm can also be reinterpreted as a smoothed analysis of the deterministic greedy algorithm. Previously, the only known bound on backlog for $p > 1$, and the only known randomized guarantees for any $p$ (including when $p = 1$), required the use of resource augmentation, meaning that the filler can only distribute at most $p(1 - ε)$ units of water in each step, and that the emptier is then permitted to remove $1 + δ$ units of water from each of $p$ cups, for some $ε, δ> 0$.
Machine-Learning Beam Tracking and Weight Optimization for mmWave Multi-UAV Links
Venue: arXiv cs.IT 2019
Year: 2019

Millimeter-wave (mmWave) hybrid analog-digital beamforming is a promising approach to satisfy the low-latency constraint in multiple unmanned aerial vehicles (UAVs) systems, which serve as network infrastructure for flexible deployment. However, in highly dynamic multi-UAV environments, analog beam tracking becomes a critical challenge. The overhead of additional pilot transmission at the price of spectral efficiency is shown necessary to achieve high resilience in operation. An efficient method to deal with high dynamics of UAVs applies machine learning, particularly Q-learning, to analog beam tracking. The proposed Q-learning-based beam tracking scheme uses current/past observations to design rewards from environments to facilitate prediction, which significantly increases the efficiency of data transmission and beam switching. Given the selected analog beams, the goal of digital beamforming is to maximize the SINR. The received pilot signals are utilized to approximate the desired signal and interference power, which yield the SINR measurements as well as the optimal digital weights. Since the selected analog beams based on the received power do not guarantee the hybrid beamforming achieving the maximization SINR, we therefore reserve additional analog beams as candidates during the beam tracking. The combination of analog beams with their digital weights achieving the maximum SINR consequently provides the optimal solution to the hybrid beamforming.
A framework for verifying deadlock and nondeterminism in UML activity diagrams based on CSP
Venue: arXiv cs.SE 2019
Year: 2019

Deadlock and nondeterminism may become increasingly hard to detect in concurrent and distributed systems. UML activity diagrams are flowcharts that model sequential and concurrent behavior. Although the UML community widely adopts such diagrams, there is no standard approach to verify the presence of deadlock and nondeterministic behavior in activity diagrams. Nondeterminism is usually neglected in the literature even though it may be considered a very relevant property. This work proposes a framework for the automatic verification of deadlock and nondeterminism in UML activity diagrams. It introduces a compositional CSP semantics for activity diagrams that is used to automatically generate CSP specifications from UML models. These specifications are the input for the automatic verification of deadlock and nondeterministic behavior using the FDR refinement checker. We propose a plugin for the Astah modeling environment that mechanizes the translation process, and that calls FDR in the background to perform the verification of properties. The tool keeps the traceability between a diagram and its CSP specification. It parses the FDR results to highlight the diagram paths that lead to a deadlock or a nondeterministic behavior. This framework adds verification capabilities to the UML modeling tool and keeps the formal semantics transparent to the users. Therefore, the user does not need to understand or manipulate formal notations during modeling. We present the results of a case study that applies the proposed framework for the verification of models in the domain of cloud computing. We discuss future applications due to the potential of our approach.
Outliagnostics: Visualizing Temporal Discrepancy in Outlying Signatures of Data Entries
Venue: arXiv cs.HC 2019
Year: 2019

This paper presents an approach to analyzing two-dimensional temporal datasets focusing on identifying observations that are significant in calculating the outliers of a scatterplot. We also propose a prototype, called Outliagnostics, to guide users when interactively exploring abnormalities in large time series. Instead of focusing on detecting outliers at each time point, we monitor and display the discrepant temporal signatures of each data entry concerning the overall distributions. Our prototype is designed to handle these tasks in parallel to improve performance. To highlight the benefits and performance of our approach, we illustrate and validate the use of Outliagnostics on real-world datasets of various sizes in different parallelism configurations. This work also discusses how to extend these ideas to handle time series with a higher number of dimensions and provides a prototype for this type of datasets.
Scalable High Performance SDN Switch Architecture on FPGA for Core Networks
Venue: arXiv cs.AR 2019
Year: 2019

Due to the increasing heterogeneity in network user requirements, dynamically varying day to day network traffic patterns and delay in-network service deployment, there is a huge demand for scalability and flexibility in modern networking infrastructure, which in return has paved way for the introduction of Software Defined Networking (SDN) in core networks. In this paper, we present an FPGA-based switch that is fully compliant with OpenFlow; the pioneering protocol for southbound interface of SDN. The switch architecture is completely implemented on hardware. The design consists of an OpenFlow Southbound agent which can process OpenFlow packets at a rate of 10Gbps. The proposed architecture speed scales up to 400Gbps while it consumes only 60% of resources on a Xilinx Virtex-7 featuring XC7VX485T FPGA.
ON-TRAC Consortium End-to-End Speech Translation Systems for the IWSLT 2019 Shared Task
Venue: arXiv cs.CL 2019
Year: 2019

This paper describes the ON-TRAC Consortium translation systems developed for the end-to-end model task of IWSLT Evaluation 2019 for the English-to-Portuguese language pair. ON-TRAC Consortium is composed of researchers from three French academic laboratories: LIA (Avignon Université), LIG (Université Grenoble Alpes), and LIUM (Le Mans Université). A single end-to-end model built as a neural encoder-decoder architecture with attention mechanism was used for two primary submissions corresponding to the two EN-PT evaluations sets: (1) TED (MuST-C) and (2) How2. In this paper, we notably investigate impact of pooling heterogeneous corpora for training, impact of target tokenization (characters or BPEs), impact of speech input segmentation and we also compare our best end-to-end model (BLEU of 26.91 on MuST-C and 43.82 on How2 validation sets) to a pipeline (ASR+MT) approach.
RBED: Reward Based Epsilon Decay
Venue: arXiv cs.AI 2019
Year: 2019

$\varepsilon$-greedy is a policy used to balance exploration and exploitation in many reinforcement learning setting. In cases where the agent uses some on-policy algorithm to learn optimal behaviour, it makes sense for the agent to explore more initially and eventually exploit more as it approaches the target behaviour. This shift from heavy exploration to heavy exploitation can be represented as decay in the $\varepsilon$ value, where $\varepsilon$ depicts the how much an agent is allowed to explore. This paper proposes a new approach to this $\varepsilon$ decay where the decay is based on feedback from the environment. This paper also compares and contrasts one such approach based on rewards and compares it against standard exponential decay. The new approach, in the environments tested, produces more consistent results that on average perform better.
Breaking and (Partially) Fixing Provably Secure Onion Routing
Venue: arXiv cs.CR 2019
Year: 2019

After several years of research on onion routing, Camenisch and Lysyanskaya, in an attempt at rigorous analysis, defined an ideal functionality in the universal composability model, together with properties that protocols have to meet to achieve provable security. A whole family of systems based their security proofs on this work. However, analyzing HORNET and Sphinx, two instances from this family, we show that this proof strategy is broken. We discover a previously unknown vulnerability that breaks anonymity completely, and explain a known one. Both should not exist if privacy is proven correctly. In this work, we analyze and fix the proof strategy used for this family of systems. After proving the efficacy of the ideal functionality, we show how the original properties are flawed and suggest improved, effective properties in their place. Finally, we discover another common mistake in the proofs. We demonstrate how to avoid it by showing our improved properties for one protocol, thus partially fixing the family of provably secure onion routing protocols.
Let Me Know What to Ask: Interrogative-Word-Aware Question Generation
Venue: arXiv cs.CL 2019
Year: 2019

Question Generation (QG) is a Natural Language Processing (NLP) task that aids advances in Question Answering (QA) and conversational assistants. Existing models focus on generating a question based on a text and possibly the answer to the generated question. They need to determine the type of interrogative word to be generated while having to pay attention to the grammar and vocabulary of the question. In this work, we propose Interrogative-Word-Aware Question Generation (IWAQG), a pipelined system composed of two modules: an interrogative word classifier and a QG model. The first module predicts the interrogative word that is provided to the second module to create the question. Owing to an increased recall of deciding the interrogative words to be used for the generated questions, the proposed model achieves new state-of-the-art results on the task of QG in SQuAD, improving from 46.58 to 47.69 in BLEU-1, 17.55 to 18.53 in BLEU-4, 21.24 to 22.33 in METEOR, and from 44.53 to 46.94 in ROUGE-L.
Quantum Optical Experiments Modeled by Long Short-Term Memory
Venue: arXiv cs.LG 2019
Year: 2019

We demonstrate how machine learning is able to model experiments in quantum physics. Quantum entanglement is a cornerstone for upcoming quantum technologies such as quantum computation and quantum cryptography. Of particular interest are complex quantum states with more than two particles and a large number of entangled quantum levels. Given such a multiparticle high-dimensional quantum state, it is usually impossible to reconstruct an experimental setup that produces it. To search for interesting experiments, one thus has to randomly create millions of setups on a computer and calculate the respective output states. In this work, we show that machine learning models can provide significant improvement over random search. We demonstrate that a long short-term memory (LSTM) neural network can successfully learn to model quantum experiments by correctly predicting output state characteristics for given setups without the necessity of computing the states themselves. This approach not only allows for faster search but is also an essential step towards automated design of multiparticle high-dimensional quantum experiments using generative machine learning models.
Closed-Form Whittle's Index-Enabled Random Access for Timely Status Update
Venue: arXiv cs.NI 2019
Year: 2019

We consider a star-topology wireless network for status update where a central node collects status data from a large number of distributed machine-type terminals that share a wireless medium. The Age of Information (AoI) minimization scheduling problem is formulated by the restless multi-armed bandit. A widely-proven near-optimal solution, i.e., the Whittle's index, is derived in closed-form and the corresponding indexability is established. The index is then generalized to incorporate stochastic, periodic packet arrivals and unreliable channels. Inspired by the index scheduling policies which achieve near-optimal AoI but require heavy signaling overhead, a contention-based random access scheme, namely Index-Prioritized Random Access (IPRA), is further proposed. Based on IPRA, terminals that are not urgent to update, indicated by their indices, are barred access to the wireless medium, thus improving the access timeliness. A computer-based simulation shows that IPRA's performance is close to the optimal AoI in this setting and outperforms standard random access schemes. Also, for applications with hard AoI deadlines, we provide reliable deadline guarantee analysis. Closed-form achievable AoI stationary distributions under Bernoulli packet arrivals are derived such that AoI deadline with high reliability can be ensured by calculating the maximum number of supportable terminals and allocating system resources proportionally.
Towards Scalable, Efficient and Accurate Deep Spiking Neural Networks with Backward Residual Connections, Stochastic Softmax and Hybridization
Venue: arXiv cs.CV 2019
Year: 2019

Spiking Neural Networks (SNNs) may offer an energy-efficient alternative for implementing deep learning applications. In recent years, there have been several proposals focused on supervised (conversion, spike-based gradient descent) and unsupervised (spike timing dependent plasticity) training methods to improve the accuracy of SNNs on large-scale tasks. However, each of these methods suffer from scalability, latency and accuracy limitations. In this paper, we propose novel algorithmic techniques of modifying the SNN configuration with backward residual connections, stochastic softmax and hybrid artificial-and-spiking neuronal activations to improve the learning ability of the training methodologies to yield competitive accuracy, while, yielding large efficiency gains over their artificial counterparts. Note, artificial counterparts refer to conventional deep learning/artificial neural networks. Our techniques apply to VGG/Residual architectures, and are compatible with all forms of training methodologies. Our analysis reveals that the proposed solutions yield near state-of-the-art accuracy with significant energy-efficiency and reduced parameter overhead translating to hardware improvements on complex visual recognition tasks, such as, CIFAR10, Imagenet datatsets.
Parallel computing in automation of decoupled fluid-thermostructural simulation approach
Venue: arXiv cs.DC 2019
Year: 2019

Decoupling approach presents a novel solution/alternative to the highly time-consuming fluid-thermal-structural simulation procedures when thermal effects and resultant displacements on machine tools are analyzed. Using high dimensional Characteristic Diagrams (CDs) along with a Clustering Algorithm that immensely reduces the data needed for training, a limited number of CFD simulations can suffice in effectively decoupling fluid and thermal-structural simulations. This approach becomes highly significant when complex geometries or dynamic components are considered. However, there is still scope for improvement in the reduction of time needed to train CDs. Parallel computation can be effectively utilized in decoupling approach in simultaneous execution of (i) CFD simulations and data export, and (ii) Clustering technique involving Genetic Algorithm and Radial Basis Function interpolation, which clusters and optimizes the training data for CDs. Parallelization reduces the entire computation duration from several days to a few hours and thereby, improving the efficiency and ease-of-use of decoupling simulation approach.