Reviewerly/exHarmony huggingface dataset:
Cross-Lingual Dependency Parsing with Unlabeled Auxiliary Languages

Cross-lingual transfer learning has become an important weapon to battle the unavailability of annotated resources for low-resource languages.One of the fundamental techniques to transfer across languages is learning language-agnostic representations, in the form of word embeddings or contextual encodings.In this work, we propose to leverage unannotated sentences from auxiliary languages to help learning language-agnostic representations.Specifically, we explore adversarial training for learning contextual encoders that produce invariant representations across languages to facilitate cross-lingual transfer.We conduct experiments on cross-lingual dependency parsing where we train a dependency parser on a source language and transfer it to a wide range of target languages.Experiments on 28 target languages demonstrate that adversarial training significantly improves the overall transfer performances under several different settings.We conduct a careful analysis to evaluate the language-agnostic representations resulted from adversarial training.
Neural Networks with a Redundant Representation: Detecting the Undetectable

We consider a three-layer Sejnowski machine and show that features learnt via contrastive divergence have a dual representation as patterns in a dense associative memory of order $P=4$. The latter is known to be able to Hebbian store an amount of patterns scaling as ${N}^{P\ensuremath{-}1}$, where $N$ denotes the number of constituting binary neurons interacting $P$ wisely. We also prove that, by keeping the dense associative network far from the saturation regime (namely, allowing for a number of patterns scaling only linearly with $N$, while $P&gt;2$) such a system is able to perform pattern recognition far below the standard signal-to-noise threshold. In particular, a network with $P=4$ is able to retrieve information whose intensity is $\mathcal{O}(1)$ even in the presence of a noise $\mathcal{O}(\sqrt{N})$ in the large $N$ limit. This striking skill stems from a redundancy representation of patterns---which is afforded given the (relatively) low-load information storage---and it contributes to explain the impressive abilities in pattern recognition exhibited by new-generation neural networks. The whole theory is developed rigorously, at the replica symmetric level of approximation, and corroborated by signal-to-noise analysis and Monte Carlo simulations.
Reading Answers on Stack Overflow: Not Enough!

Stack Overflow is one of the most active communities for developers to share their programming knowledge. Answers posted on Stack Overflow help developers solve issues during software development. In addition to posting answers, users can also post comments to further discuss their associated answers. As of Aug 2017, there are 32.3 million comments that are associated with answers, forming a large collection of crowdsourced repository of knowledge on top of the commonly-studied Stack Overflow answers. In this study, we wish to understand how the commenting activities contribute to the crowdsourced knowledge. We investigate what users discuss in comments, and analyze the characteristics of the commenting dynamics, (i.e., the timing of commenting activities and the roles of commenters). We find that: 1) the majority of comments are informative and thus can enhance their associated answers from a diverse range of perspectives. However, some comments contain content that is discouraged by Stack Overflow. 2) The majority of commenting activities occur after the acceptance of an answer. More than half of the comments are fast responses occurring within one day of the creation of an answer, while later comments tend to be more informative. Most comments are rarely integrated back into their associated answers, even though such comments are informative. 3) Insiders (i.e., users who posted questions/answers before posting a comment in a question thread) post the majority of comments within one month, and outsiders (i.e., users who never posted any question/answer before posting a comment) post the majority of comments after one month. Inexperienced users tend to raise limitations and concerns while experienced users tend to enhance the answer through commenting. Our study provides insights into the commenting activities in terms of their content, timing, and the individuals who perform the commenting. For the purpose of long-term knowledge maintenance and effective information retrieval for developers, we also provide actionable suggestions to encourage Stack Overflow users/engineers/moderators to leverage our insights for enhancing the current Stack Overflow commenting system for improving the maintenance and organization of the crowdsourced knowledge.
Hybrid adversarial network for unsupervised domain adaptation

Recent advances suggest that adversarial domain adaptation has been embedding into deep neural networks to learn domain-transferable representations, which reduces distribution divergence in both the training and test samples. However, previous adversarial learning algorithms only resort to learn domain-transferable feature representation by bounding the feature distribution discrepancy cross-domain. These approaches, however, may lead to misalignment and poor generalization results due to without further exploiting class information and task-special adaptation. To cope with this issue, a joint adversarial learning with class information and domain alignment deep network architecture, is proposed which is named Hybrid Adversarial Network (HAN). Specifically, it incorporates a classification loss to learn a discriminative classifier, and a domain adversarial network learns a domain-transferable representation to reduce domain shift. Meanwhile, a CORAL loss is used to minimize the discrepancy between the covariance matrices in the two domains. Additionally, we introduce an adaptation layer for further boosting the performance of HAN model. Comprehensive cross-domain visual recognition experiments validate that our method exceeds the state-of-the-art methods on three real-world benchmark including Office-31, Office-Home, and ImageCLEF-DA datasets.
EchoFace: Acoustic Sensor-Based Media Attack Detection for Face Authentication

Face authentication systems have gained widespread popularity because of their user-friendly usage and increasing recognition accuracy. Unfortunately, the boom in mobile social networks has bought with it media-based facial forgery; a critical threat where an adversary forges or replays the victim's photograph/video to fool the system. In this article, we propose EchoFace, an effective and robust liveness detection system to enhance face authentication in defending against media-based attacks, which works with today's smartphones/smartwatches without any hardware modification. EchoFace uses active acoustic sensing to differentiate the uneven stereostructure of the face and the flat forged media. Our proposed scheme effectively extracts the desired reflection profiles from the target. Moreover, we propose effective similarity measurements of reflection profiles to distinguish live users from forged media, which works robustly under various environmental conditions. EchoFace only requires low cost and universally equipped acoustic sensors without human intervention for liveness detection, which can be easily deployed in a variety of application scenarios. We implement EchoFace on commercial smartphones, and experiment results show that EchoFace achieves an average detection accuracy higher than 96% and false alarm rate lower than 4% across various media attacks and different levels of background noise. This shows its great potential to enhance the security of widely deployed face authentication systems in real scenarios.
La llegada de la inteligencia artificial a la educación

Every day the use of artificial intelligence in our daily life becomes increasingly evident, however there are still many elements to work and organize before we can make a true implementation of artificial intelligence in education, even so it is still a great advantage and help when imparting and generating knowledge this added to the challenges of the 21st century that seek an integrality and a true transversality of technology and in the various axes of knowledge, this document is intended to reflect on the importance and true utility of the implementation and assistance of AI in our teaching work that also allows us to see clear examples worldwide on digital literacy that aims to understand more in depth about the true usefulness and practicality of AI, also to focus and build true pedagogical skills aimed at building a scientific thought and technological.
A Novel Learning Rate Function and Its Application on the SVD++ Recommendation Algorithm

The recommendation algorithm based on Singular Value Decomposition (SVD)++ is a widely used algorithm for its good prediction performance. However, with the rapid increase of data in smart societies, the poor computational performance of the SVD++ recommendation algorithm becomes a prominent disadvantage, for it takes a longer time to optimize the objective function during constructing the prediction model. The learning rate function is a significant factor in the prediction model based on the SVD++ recommendation algorithm. It can directly affect the convergence speed of the prediction model and the performance of the model. The traditional model uses an exponential function, natural exponential function or piecewise constant as its learning rate function. In this paper, a novel adaptive learning rate (ALR) function is proposed, which combines the exponential with linear functions, and the function is applied to the SVD++ recommendation algorithm. The highlights of the paper are as follows. First, with a larger initial value, the proposed function descends quicker and tends to the end with a less step. Second, the theoretical properties of the proposed learning rate function are verified through theoretical analysis, including the theoretical proof of its convergence and the iteration speed comparison. Compared to the existing learning rate functions, the proposed ALR function works better on the convergence speed through mathematical derivation. Finally, the novel ALR function is applied to the SVD++ recommendation algorithm as recommendation model ALRSVD++. Some existing learning rate methods are used as benchmarks for illustrating the computation and prediction performances of proposed ALR function and its ALRSVD++ model. Experimental results demonstrated that the SVD++ recommendation algorithm based on the proposed ALR function improved computational efficiency of the training model ALRSVD++ significantly. Especially, to the larger size training dataset, the iterations and training time based on the proposed ALR function and ALRSVD++ model reduced in a great deal, without greatly sacrificing the recommendation performance.
DGCN: Dynamic Graph Convolutional Network for Efficient Multi-Person Pose Estimation

Multi-person pose estimation aims to detect human keypoints from images with multiple persons. Bottom-up methods for multi-person pose estimation have attracted extensive attention, owing to the good balance between efficiency and accuracy. Recent bottom-up methods usually follow the principle of keypoints localization and grouping, where relations between keypoints are the keys to group keypoints. These relations spontaneously construct a graph of keypoints, where the edges represent the relations between two nodes (i.e., keypoints). Existing bottom-up methods mainly define relations by empirically picking out edges from this graph, while omitting edges that may contain useful semantic relations. In this paper, we propose a novel Dynamic Graph Convolutional Module (DGCM) to model rich relations in the keypoints graph. Specifically, we take into account all relations (all edges of the graph) and construct dynamic graphs to tolerate large variations of human pose. The DGCM is quite lightweight, which allows it to be stacked like a pyramid architecture and learn structural relations from multi-level features. Our network with single DGCM based on ResNet-50 achieves relative gains of 3.2% and 4.8% over state-of-the-art bottom-up methods on COCO keypoints and MPII dataset, respectively.
Relation-Guided Spatial Attention and Temporal Refinement for Video-Based Person Re-Identification

Video-based person re-identification has received considerable attention in recent years due to its significant application in video surveillance. Compared with image-based person re-identification, video-based person re-identification is characterized by a much richer context, which raises the significance of identifying informative regions and fusing the temporal information across frames. In this paper, we propose two relation-guided modules to learn reinforced feature representations for effective re-identification. First, a relation-guided spatial attention (RGSA) module is designed to explore the discriminative regions globally. The weight at each position is determined by its feature as well as the relation features from other positions, revealing the dependence between local and global contents. Based on the adaptively weighted frame-level feature, then, a relation-guided temporal refinement (RGTR) module is proposed to further refine the feature representations across frames. The learned relation information via the RGTR module enables the individual frames to complement each other in an aggregation manner, leading to robust video-level feature representations. Extensive experiments on four prevalent benchmarks verify the state-of-the-art performance of the proposed method.
MDP-Based Quantitative Analysis Framework for Proof of Authority

The widespread recognition of the blockchain technology that decoupled from the cryptocurrency has raised the interest of the industry. As the apparent drawbacks of the Proof of Work (PoW) protocol hinder the large deployment of PoW protocol for many distributed applications, the consensus mechanism evolved quickly to satisfy the business requirements. Since the consensus mechanism is the core of the blockchain technology, it is imperative to ensure that it performs correctly. Proof of Authority, a novel Byzantine fault-tolerant algorithm, is an appealing alternative to PoW algorithm for its high throughput rate and the security. However, the PoA protocol seems to do not convince the public about its security as it lacks adequate analysis. In this paper, we firstly explored the balance between security and scalability of PoA protocol, and subsequently, we proposed a flexible quantitative analysis framework based on Markov Decision Process. The framework can be used to indicate the security from the perspective of the adversarial optimal strategies. To capture the action strategies of an adversary, we also presented a formal model of PoA and its attack model. Since the modeling process for a blockchain protocol is not a trivial task, we implemented a case study to demonstrate the modeling process and discussed the security provisions of a specific PoA protocol.
MetaMT, a Meta Learning Method Leveraging Multiple Domain Data for Low Resource Machine Translation

Neural machine translation (NMT) models have achieved state-of-the-art translation quality with a large quantity of parallel corpora available. However, their performance suffers significantly when it comes to domain-specific translations, in which training data are usually scarce. In this paper, we present a novel NMT model with a new word embedding transition technique for fast domain adaption. We propose to split parameters in the model into two groups: model parameters and meta parameters. The former are used to model the translation while the latter are used to adjust the representational space to generalize the model to different domains. We mimic the domain adaptation of the machine translation model to low-resource domains using multiple translation tasks on different domains. A new training strategy based on meta-learning is developed along with the proposed model to update the model parameters and meta parameters alternately. Experiments on datasets of different domains showed substantial improvements of NMT performances on a limited amount of data.
Probabilistic simulation of quantum circuits using a deep-learning architecture

The fundamental question of how to best simulate quantum systems using conventional computational resources lies at the forefront of condensed matter and quantum computation. It impacts both our understanding of quantum materials and our ability to emulate quantum circuits. Here we present an exact formulation of quantum dynamics via factorized generalized measurements which maps quantum states to probability distributions with the advantage that local unitary dynamics and quantum channels map to local quasistochastic matrices. This representation provides a general framework for using state-of-the-art probabilistic models in machine learning for the simulation of quantum many-body dynamics. Using this framework, we have developed a practical algorithm to simulate quantum circuits using an attention network based on a powerful neural network ansatz responsible for the most recent breakthroughs in natural language processing. We demonstrate our approach by simulating circuits that build Greenberger-Horne-Zeilinger and linear graph states of up to 60 qubits, as well as a variational quantum eigensolver circuit for preparing the ground state of the transverse field Ising model on several system sizes. Our methodology constitutes a modern machine learning approach to the simulation of quantum physics with applicability both to quantum circuits as well as other quantum many-body systems.
Geometry-Driven Self-Supervised Method for 3D Human Pose Estimation

The neural network based approach for 3D human pose estimation from monocular images has attracted growing interest. However, annotating 3D poses is a labor-intensive and expensive process. In this paper, we propose a novel self-supervised approach to avoid the need of manual annotations. Different from existing weakly/self-supervised methods that require extra unpaired 3D ground-truth data to alleviate the depth ambiguity problem, our method trains the network only relying on geometric knowledge without any additional 3D pose annotations. The proposed method follows the two-stage pipeline: 2D pose estimation and 2D-to-3D pose lifting. We design the transform re-projection loss that is an effective way to explore multi-view consistency for training the 2D-to-3D lifting network. Besides, we adopt the confidences of 2D joints to integrate losses from different views to alleviate the influence of noises caused by the self-occlusion problem. Finally, we design a two-branch training architecture, which helps to preserve the scale information of re-projected 2D poses during training, resulting in accurate 3D pose predictions. We demonstrate the effectiveness of our method on two popular 3D human pose datasets, Human3.6M and MPI-INF-3DHP. The results show that our method significantly outperforms recent weakly/self-supervised approaches.
Spatiotemporally Constrained Action Space Attacks on Deep Reinforcement Learning Agents

Robustness of Deep Reinforcement Learning (DRL) algorithms towards adversarial attacks in real world applications such as those deployed in cyber-physical systems (CPS) are of increasing concern. Numerous studies have investigated the mechanisms of attacks on the RL agent's state space. Nonetheless, attacks on the RL agent's action space (corresponding to actuators in engineering systems) are equally perverse, but such attacks are relatively less studied in the ML literature. In this work, we first frame the problem as an optimization problem of minimizing the cumulative reward of an RL agent with decoupled constraints as the budget of attack. We propose the white-box Myopic Action Space (MAS) attack algorithm that distributes the attacks across the action space dimensions. Next, we reformulate the optimization problem above with the same objective function, but with a temporally coupled constraint on the attack budget to take into account the approximated dynamics of the agent. This leads to the white-box Look-ahead Action Space (LAS) attack algorithm that distributes the attacks across the action and temporal dimensions. Our results showed that using the same amount of resources, the LAS attack deteriorates the agent's performance significantly more than the MAS attack. This reveals the possibility that with limited resource, an adversary can utilize the agent's dynamics to malevolently craft attacks that causes the agent to fail. Additionally, we leverage these attack strategies as a possible tool to gain insights on the potential vulnerabilities of DRL agents.
Efficient Inference of Optimal Decision Trees

Inferring a decision tree from a given dataset is a classic problem in machine learning. This problem consists of building, from a labelled dataset, a tree where each node corresponds to a class and a path between the tree root and a leaf corresponds to a conjunction of features to be satisfied in this class. Following the principle of parsimony, we want to infer a minimal tree consistent with the dataset. Unfortunately, inferring an optimal decision tree is NP-complete for several definitions of optimality. For this reason, the majority of existing approaches rely on heuristics, and the few existing exact approaches do not work on large datasets. In this paper, we propose a novel approach for inferring an optimal decision tree with a minimum depth based on the incremental generation of Boolean formulas. The experimental results indicate that it scales sufficiently well and the time it takes to run grows slowly with the size of datasets.
A Partially Amended Hybrid Bi-GRU—ARIMA Model (PAHM) for Predicting Solar Irradiance in Short and Very-Short Terms

Solar renewable energy (SRE) applications are substantial in eradicating the rising global energy shortages and reversing the approaching environmental apocalypse. Hence, effective solar irradiance forecasting models are crucial in utilizing SRE efficiently. This paper introduces a partially amended hybrid model (PAHM) by the implementation of a new algorithm. The algorithm innovatively utilizes bi-directional gated unit (Bi-GRU), autoregressive integrated moving average (ARIMA) and naive decomposition models to predict solar irradiance in 5-min and 60-min intervals. Meanwhile, the models’ generalizability strengths would be tested under an 11-fold cross-validation and are further classified according to their computational costs. The dataset consists of 32 months’ solar irradiance and weather conditions records. A fundamental result of this study was that the single models (Bi-GRU and ARIMA) outperformed the hybrid models (PAHM, classical hybrid model) in the 5-min predictions, negating the assumptions that hybrid models oust single models in every time interval. PAHM provided the highest accuracy level in the 60-min predictions and improved the accuracy levels of the classical hybrid model by 5%, on average. The single models were rigorous under the 11-fold cross-validation, performing well with different datasets; although the computational efficiency of the Bi-GRU model was, by far, the best among the models.