Reviewerly/exHarmony huggingface dataset:
Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations

Sparse language vectors from linguistic typology databases and learned embeddings from tasks like multilingual machine translation have been investigated in isolation, without analysing how they could benefit from each other's language characterisation.We propose to fuse both views using singular vector canonical correlation analysis and study what kind of information is induced from each source.By inferring typological features and language phylogenies, we observe that our representations embed typology and strengthen correlations with language relationships.We then take advantage of our multi-view language vector space for multilingual machine translation, where we achieve competitive overall translation accuracy in tasks that require information about language similarities, such as language clustering and ranking candidates for multilingual transfer.With our method, which is also released as a tool, we can easily project and assess new languages without expensive retraining of massive multilingual or ranking models, which are major disadvantages of related approaches.
Multilingual Knowledge Graph Completion via Ensemble Knowledge Transfer

Predicting missing facts in a knowledge graph (KG) is a crucial task in knowledge base construction and reasoning, and it has been the subject of much research in recent works using KG embeddings.While existing KG embedding approaches mainly learn and predict facts within a single KG, a more plausible solution would benefit from the knowledge in multiple language-specific KGs, considering that different KGs have their own strengths and limitations on data quality and coverage.This is quite challenging, since the transfer of knowledge among multiple independently maintained KGs is often hindered by the insufficiency of alignment information and the inconsistency of described facts.In this paper, we propose KEnS, a novel framework for embedding learning and ensemble knowledge transfer across a number of languagespecific KGs.KEnS embeds all KGs in a shared embedding space, where the association of entities is captured based on selflearning.Then, KEnS performs ensemble inference to combine prediction results from embeddings of multiple language-specific KGs, for which multiple ensemble techniques are investigated.Experiments on five real-world language-specific KGs show that KEnS consistently improves state-of-the-art methods on KG completion, via effectively identifying and leveraging complementary knowledge.
Dimensional Quantum Memory Advantage in the Simulation of Stochastic Processes

Stochastic processes underlie a vast range of natural and social phenomena. Some processes such as atomic decay feature intrinsic randomness, whereas other complex processes, e.g., traffic congestion, are effectively probabilistic because we cannot track all relevant variables. To simulate a stochastic system’s future behavior, information about its past must be stored, and thus memory is a key resource. Quantum information processing promises a memory advantage for stochastic simulation. Here, we report the first experimental demonstration that a quantum stochastic simulator can encode the required information in fewer dimensions than any classical simulator, thereby achieving a quantum advantage in minimal memory requirements using an individual simulator. This advantage is in contrast to recent proof-of-concept experiments, where the memory saving would only become accessible in the limit of a large number of parallel simulations. In those examples, the minimal memory registers of individual quantum simulators had the same dimensionality as their classical counterparts. Our photonic experiment thus establishes the potential of new, practical resource savings in the simulation of complex systems.Received 15 March 2019Revised 18 September 2019DOI:https://doi.org/10.1103/PhysRevX.9.041013Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasQuantum information processingQuantum simulationQuantum Information
PGM-Explainer: Probabilistic Graphical Model Explanations for Graph Neural Networks

In Graph Neural Networks (GNNs), the graph structure is incorporated into the learning of node representations. This complex structure makes explaining GNNs' predictions become much more challenging. In this paper, we propose PGM-Explainer, a Probabilistic Graphical Model (PGM) model-agnostic explainer for GNNs. Given a prediction to be explained, PGM-Explainer identifies crucial graph components and generates an explanation in form of a PGM approximating that prediction. Different from existing explainers for GNNs where the explanations are drawn from a set of linear functions of explained features, PGM-Explainer is able to demonstrate the dependencies of explained features in form of conditional probabilities. Our theoretical analysis shows that the PGM generated by PGM-Explainer includes the Markov-blanket of the target prediction, i.e. including all its statistical information. We also show that the explanation returned by PGM-Explainer contains the same set of independence statements in the perfect map. Our experiments on both synthetic and real-world datasets show that PGM-Explainer achieves better performance than existing explainers in many benchmark tasks.
Semi-device-independent certification of indefinite causal order

When transforming pairs of independent quantum operations according to the fundamental rules of quantum theory, an intriguing phenomenon emerges: some such higher-order operations may act on the input operations in an indefinite causal order. Recently, the formalism of process matrices has been developed to investigate these noncausal properties of higher-order operations. This formalism predicts, in principle, statistics that ensure indefinite causal order even in a device-independent scenario, where the involved operations are not characterised. Nevertheless, all physical implementations of process matrices proposed so far require full characterisation of the involved operations in order to certify such phenomena. Here we consider a semi-device-independent scenario, which does not require all operations to be characterised. We introduce a framework for certifying noncausal properties of process matrices in this intermediate regime and use it to analyse the quantum switch, a well-known higher-order operation, to show that, although it can only lead to causal statistics in a device-independent scenario, it can exhibit noncausal properties in semi-device-independent scenarios. This proves that the quantum switch generates stronger noncausal correlations than it was previously known.
Adversarial Attack and Defense of Structured Prediction Models

Building an effective adversarial attacker and elaborating on countermeasures for adversarial attacks for natural language processing (NLP) have attracted a lot of research in recent years.However, most of the existing approaches focus on classification problems.In this paper, we investigate attacks and defenses for structured prediction tasks in NLP.Besides the difficulty of perturbing discrete words and the sentence fluency problem faced by attackers in any NLP tasks, there is a specific challenge to attackers of structured prediction models: the structured output of structured prediction models is sensitive to small perturbations in the input.To address these problems, we propose a novel and unified framework that learns to attack a structured prediction model using a sequence-to-sequence model with feedbacks from multiple reference models of the same structured prediction task.Based on the proposed attack, we further reinforce the victim model with adversarial training, making its prediction more robust and accurate.We evaluate the proposed framework in dependency parsing and part-of-speech tagging.Automatic and human evaluations show that our proposed framework succeeds in both attacking state-of-the-art structured prediction models and boosting them with adversarial training.
Identifying Incorrect Labels in the CoNLL-2003 Corpus

The CoNLL-2003 corpus for Englishlanguage named entity recognition (NER) is one of the most influential corpora for NER model research.A large number of publications, including many landmark works, have used this corpus as a source of ground truth for NER tasks.In this paper, we examine this corpus and identify over 1300 incorrect labels (out of 35089 in the corpus).In particular, the number of incorrect labels in the test fold is comparable to the number of errors that state-of-the-art models make when running inference over this corpus.We describe the process by which we identified these incorrect labels, using novel variants of techniques from semi-supervised learning.We also summarize the types of errors that we found, and we revisit several recent results in NER in light of the corrected data.Finally, we show experimentally that our corrections to the corpus have a positive impact on three state-ofthe-art models.
Pixel-Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation

Domain adaptive semantic segmentation aims to train a model performing satisfactory pixel-level predictions on the target with only out-of-domain (source) annotations. The conventional solution to this task is to minimize the discrepancy between source and target to enable effective knowledge transfer. Previous domain discrepancy minimization methods are mainly based on the adversarial training. They tend to consider the domain discrepancy globally, which ignore the pixel-wise relationships and are less discriminative. In this paper, we propose to build the pixel-level cycle association between source and target pixel pairs and contrastively strengthen their connections to diminish the domain gap and make the features more discriminative. To the best of our knowledge, this is a new perspective for tackling such a challenging task. Experiment results on two representative domain adaptation benchmarks, i.e. GTAV $\rightarrow$ Cityscapes and SYNTHIA $\rightarrow$ Cityscapes, verify the effectiveness of our proposed method and demonstrate that our method performs favorably against previous state-of-the-arts. Our method can be trained end-to-end in one stage and introduces no additional parameters, which is expected to serve as a general framework and help ease future research in domain adaptive semantic segmentation. Code is available at this https URL Level-Cycle-Association.
CLUE: A Fast Parallel Clustering Algorithm for High Granularity Calorimeters in High-Energy Physics

One of the challenges of high granularity calorimeters, such as that to be built to cover the endcap region in the CMS Phase-2 Upgrade for HL-LHC, is that the large number of channels causes a surge in the computing load when clustering numerous digitized energy deposits (hits) in the reconstruction stage. In this article, we propose a fast and fully parallelizable density-based clustering algorithm, optimized for high-occupancy scenarios, where the number of clusters is much larger than the average number of hits in a cluster. The algorithm uses a grid spatial index for fast querying of neighbors and its timing scales linearly with the number of hits within the range considered. We also show a comparison of the performance on CPU and GPU implementations, demonstrating the power of algorithmic parallelization in the coming era of heterogeneous computing in high-energy physics.
Towards Pentesting Automation Using the Metasploit Framework

Penetration testing is a well known methodology assessing security vulnerabilities by executing complex steps which form an attack. Professional pentesting is an expensive service that sometimes cannot fit in the budget of Small and Medium Enterprises. Automating this process means it can be executed even by inexperienced system administrators while it saves time for professionals. The difficulty of this problem consists in the heterogeneity of networks and systems so the techniques need to be adapted each time. Our approach is based on identifying system characteristics, search for existing vulnerabilities and applying machine learning for selecting the most appropriate exploit. The model was trained using data collected from exploited machines on the “Hack the Box” learning platform and delivers exploits from the Metasploit framework. The evaluation shows that the proposed framework can exploit a fair number of systems and can be extended to support new classes of exploits and new pentesting methodologies.
An Evolutionary DenseRes Deep Convolutional Neural Network for Medical Image Segmentation

The performance of a Convolutional Neural Network (CNN) highly depends on its architecture and corresponding parameters. Manually designing a CNN is a time-consuming process in regards to the various layers that it can have, and the variety of parameters that must be set up. Increasing the complexity of the network structure by employing various types of connections makes designing a network even more challenging. Evolutionary computation as an optimisation technique can be applied to arrange the CNN layers and/or initiate its parameters automatically or semi-automatically. Dense network and Residual network are two popular network structures that were introduced to facilitate the training of deep networks. In this paper, leveraging the potentials of Dense and Residual blocks, and using the capability of evolutionary computation, we propose an automatic evolutionary model to detect an optimum and accurate network structure and its parameters for medical image segmentation. The proposed evolutionary DenseRes model is employed for segmentation of six publicly available MRI and CT medical datasets. The proposed model obtained high accuracy while employing networks with minimal parameters for the segmentation of medical images and outperformed manual and automatic designed networks, including U-Net, Residual U-Net, Dense U-Net, Non-Bypass Dense, NAS U-Net, AdaresU-Net, and EvoU-Net.
Unitary learning for diffractive deep neural network

Realization of deep learning with coherent diffraction has achieved remarkable development nowadays, which benefits on the fact that matrix multiplication can be optically executed in parallel with high band-with and low latency. Coherent optical field in the form of complex-valued entity can be manipulated into a task-oriented output. In this paper, a modulation mechanism is established by implementing the equivalence between a digital deep unitary neural network and optical coherent diffraction. We present a unitary learning avenue on diffractive deep neural network, meeting the physical unitary prior in coherent diffraction. The Unitary learning is a Backpropagation serving to unitary weights update through the gradient translation from Euclidean to Riemannian space. The temporal-space evolution characteristics in unitary learning are formulated and elucidated. And a compatible condition on how to select the nonlinear activation in complex space is unveiled, encapsulating the fundamental sigmoid, tanh and quasi-ReLu in complex space available in a single channel training. The performance of phase-ReLu is particularly emphasized. As a preliminary application, diffractive deep neural network with unitary learning is tentatively implemented on the 2D classification and verification tasks.
Impact of PV system orientation angle accuracy on greenhouse gases mitigation

Sun, a free source of energy, has moved the world to consider photovoltaic (PV) system as green renewable energy since fossil fuel power plant accounts for a significant share of greenhouse gas (GHG) emissions across the globe. The output of the PV system is limited to many factors; however, the acute accuracy of orientation (tilt and azimuth) angles exposes the PV surfaces to high doses of solar radiations. Which increases the PV system output; in turn GHG mitigation potential. This paper draws a relationship between accuracy of orientation angles impact on PV system's GHG mitigation potential. For reference, the accuracy of orientation angles in Pakistan's Capital increases the annual average daily solar radiations level from 4.02 (horizontal surface) to 4.39 (−33.6 tilt and 180 azimuth) kWh/m2/d. Accuracy of orientation angles on multiple end energy users (equivalent to 1 MW) PV system has potential to avoid additional 65.1 and 22.6 tonnes of CO2 equivalent and production of 158987.13 and 55,120.1 kW h compared with a same size PV system with horizontal surface and 33.6 tilt angle (equivalent to location's latitude), respectively. PV system orientation angles accuracy has the potential to avoid tonnes of GHG emissions without any investment, operation and maintenance costs.
SemEval-2020 Task 3: Graded Word Similarity in Context

This paper presents the Graded Word Similarity in Context (GWSC) task which asked participants to predict the effects of context on human perception of similarity in English, Croatian, Slovene and Finnish.We received 15 submissions and 11 system description papers.A new dataset (CoSimLex) was created for evaluation in this task: it contains pairs of words, each annotated within two short text passages.Systems beat the baselines by significant margins, but few did well in more than one language or subtask.Almost every system employed a Transformer model, but with many variations in the details: WordNet sense embeddings, translation of contexts, TF-IDF weightings, and the automatic creation of datasets for fine-tuning were all used to good effect.
A Game Theoretic Framework for Model Based Reinforcement Learning

Model-based reinforcement learning (MBRL) has recently gained immense interest due to its potential for sample efficiency and ability to incorporate off-policy data. However, designing stable and efficient MBRL algorithms using rich function approximators have remained challenging. To help expose the practical challenges in MBRL and simplify algorithm design from the lens of abstraction, we develop a new framework that casts MBRL as a game between: (1) a policy player, which attempts to maximize rewards under the learned model; (2) a model player, which attempts to fit the real-world data collected by the policy player. For algorithm development, we construct a Stackelberg game between the two players, and show that it can be solved with approximate bi-level optimization. This gives rise to two natural families of algorithms for MBRL based on which player is chosen as the leader in the Stackelberg game. Together, they encapsulate, unify, and generalize many previous MBRL algorithms. Furthermore, our framework is consistent with and provides a clear basis for heuristics known to be important in practice from prior works. Finally, through experiments we validate that our proposed algorithms are highly sample efficient, match the asymptotic performance of model-free policy gradient, and scale gracefully to high-dimensional tasks like dexterous hand manipulation. Additional details and code can be obtained from the project page at this https URL
A Method to Determine the Most Suitable Initial Conditions of Chaotic Map in Statistical Randomness Applications

The processes and systems in the real world actually contain order and symmetry. Understanding these order and symmetrical behavior has been a common effort of scientists. Chaos theory has been an interesting topic to understand these order and symmetrical behavior. One of the most important reasons for this interest is the rich dynamics of chaotic systems. A remarkable application of these systems is statistical randomness. Especially in random number generator designs based on discrete time chaotic systems, an important design parameter affecting the success of the generator (statistical randomness properties) is the initial conditions of chaotic maps. Obtaining different initial conditions that will meet the statistical requirements is important in terms of generating different random number sequences. In order to determine the initial conditions, an algorithm that updates the initial conditions depending on the number of successful statistical tests is proposed. Although the proposed design approach is similar to a back propagation neural network, it has a unique design approach. The NIST SP 800-22 test suite has been used to analyze the statistical properties of the proposed generator. It is known that the NIST SP 800-22 test suite is a hypothesis test. Therefore, in order to show the success of the proposed method in the best way, various additional analysis studies have been carried out proving that the generator outputs have a uniform distribution. Using the proposed method, six different initial conditions have been determined that provide statistical random properties for the discrete-time chaotic systems known as logistic map and tent map. 1,000,000 bits have been generated using the obtained initial conditions. These bit values are then converted to decimal values between 0-15. It is observed that the obtained numbers have a uniform distribution. These outputs are thought to be applicable in many areas such as games, simulation, modeling, determination of optimization parameters and cryptology. It is shown in a practical application for cryptographic substitution-box designs to examine the success of the outputs.
Attention R-CNN for Accident Detection

This paper addresses accident detection where we not only detect objects with classes, but also recognize their characteristic properties. More specifically, we aim at simultaneously detecting object class bounding boxes on roads and recognizing their status such as safe, dangerous, or crashed. To achieve this goal, we construct a new dataset and propose a baseline method for benchmarking the task of accident detection. We design an accident detection network, called Attention R-CNN, which consists of two streams: one is for object detection with classes and one for characteristic property computation. As an attention mechanism capturing contextual information in the scene, we integrate global contexts exploited from the scene into the stream for object detection. This introduced attention mechanism enables us to recognize object characteristic properties. Extensive experiments on the newly constructed dataset demonstrate the effectiveness of our proposed network. The dataset and source code are publicly available on our project page.