Reviewerly/exHarmony huggingface dataset:
Promoting the Knowledge of Source Syntax in Transformer NMT Is Not Needed

The utility of linguistic annotation in neural machine translation seemed to had been established in past papers. The experiments were however limited to recurrent sequence-to-sequence architectures and relatively small data settings. We focus on the state-of-the-art Transformer model and use comparably larger corpora. Specifically, we try to promote the knowledge of source-side syntax using multi-task learning either through simple data manipulation techniques or through a dedicated model component. In particular, we train one of Transformer attention heads to produce source-side dependency tree. Overall, our results cast some doubt on the utility of multi-task setups with linguistic information. The data manipulation techniques, recommended in previous works, prove ineffective in large data settings. The treatment of self-attention as dependencies seems much more promising: it helps in translation and reveals that Transformer model can very easily grasp the syntactic structure. An important but curious result is, however, that identical gains are obtained by using trivial ``linear trees'' instead of true dependencies. The reason for the gain thus may not be coming from the added linguistic knowledge but from some simpler regularizing effect we induced on self-attention matrices.
Appearance Learning for Image-Based Motion Estimation in Tomography

In tomographic imaging, anatomical structures are reconstructed by applying a pseudo-inverse forward model to acquired signals. Geometric information within this process is usually depending on the system setting only, i.e., the scanner position or readout direction. Patient motion therefore corrupts the geometry alignment in the reconstruction process resulting in motion artifacts. We propose an appearance learning approach recognizing the structures of rigid motion independently from the scanned object. To this end, we train a siamese triplet network to predict the reprojection error (RPE) for the complete acquisition as well as an approximate distribution of the RPE along the single views from the reconstructed volume in a multi-task learning approach. The RPE measures the motion-induced geometric deviations independent of the object based on virtual marker positions, which are available during training. We train our network using 27 patients and deploy a 21-4-2 split for training, validation and testing. In average, we achieve a residual mean RPE of 0.013mm with an inter-patient standard deviation of 0.022mm. This is twice the accuracy compared to previously published results. In a motion estimation benchmark the proposed approach achieves superior results in comparison with two state-of-the-art measures in nine out of twelve experiments. The clinical applicability of the proposed method is demonstrated on a motion-affected clinical dataset.
MODE-LSTM: A Parameter-efficient Recurrent Network with Multi-Scale for Sentence Classification

The central problem of sentence classification is to extract multi-scale n-gram features for understanding the semantic meaning of sentences.Most existing models tackle this problem by stacking CNN and RNN models, which easily leads to feature redundancy and overfitting because of relatively limited datasets.In this paper, we propose a simple yet effective model called Multi-scale Orthogonal inDependEnt LSTM (MODE-LSTM), which not only has effective parameters and good generalization ability, but also considers multiscale n-gram features.We disentangle the hidden state of the LSTM into several independently updated small hidden states and apply an orthogonal constraint on their recurrent matrices.We then equip this structure with sliding windows of different sizes for extracting multi-scale n-gram features.Extensive experiments demonstrate that our model achieves better or competitive performance against state-of-the-art baselines on eight benchmark datasets.We also combine our model with BERT to further boost the generalization performance.
Maximization of Site-Specific Solar Photovoltaic Energy Generation through Tilt Angle and Sun-Hours Optimization

Many photovoltaic solar projects do not achieve optimum energy and power outputs due to poor technical sizing and system design approaches. Concerns on low-conversion rates, high intermittencies, and high-capital costs still haunt PV projects. The establishment of design methodologies that would result in increased outputs from solar arrays is crucial in addressing the aforementioned issues. The tilt angles of installed PV modules are critical factors that influence the power output of solar modules. Several resources are available that provide generic linear fits and estimation of tilt angles for various global regions. However, very few are capable of determining precise, location-specific tilt angles that would allow for optimal power output and energy generation. This paper presents a methodology developed to establish the optimum tilt angles for solar panels installed at specific locations, thus ensuring maximum energy generation. The modeling is based on the maximization of the solar irradiation incident on the surface of a PV panel by considering multiple site-specific variables. Different sets of transcendent equations have been derived which were used to calculate optimum tilt angles and the subsequent energy generation from specific configurations of photovoltaic arrays. The resulting algorithms were used to determine optimum tilt angles and energy generation for solar PV installations in Athi River, Kenya. Dynamic and static optimal tilt angles were compared with the region’s baseline industry practice of using a fixed tilt angle of 15◦. It was observed that the dynamic tilt angles improved the daily solar energy output by up to 6.15%, while the computed optimal static tilt angle provided a 2.87% output increment. This improvement presents a significant impact on the technical specification of the PV system with a consequent reduction in the investment and operational cost of such installations. It further demonstrated that the use of the optimum static tilt angle results in cost and space savings of up to 2.8% as compared to the standard industry practice. Additionally, 5.8% cost and space savings were attained by the utilization of dynamic tilt angles.
Integrating Task Specific Information into Pretrained Language Models for Low Resource Fine Tuning

Pretrained Language Models (PLMs) have improved the performance of natural language understanding in recent years.Such models are pretrained on large corpora, which encode the general prior knowledge of natural languages but are agnostic to information characteristic of downstream tasks.This often results in overfitting when fine-tuned with low resource datasets where task-specific information is limited.In this paper, we integrate label information as a task-specific prior into the self-attention component of pretrained BERT models.Experiments on several benchmarks and real-word datasets suggest that the proposed approach can largely improve the performance of pretrained models when finetuning with small datasets.
Adversarial Training is a Form of Data-dependent Operator Norm Regularization

We establish a theoretical link between adversarial training and operator norm regularization for deep neural networks. Specifically, we prove that $\ell_p$-norm constrained projected gradient ascent based adversarial training with an $\ell_q$-norm loss on the logits of clean and perturbed inputs is equivalent to data-dependent (p, q) operator norm regularization. This fundamental connection confirms the long-standing argument that a network's sensitivity to adversarial examples is tied to its spectral properties and hints at novel ways to robustify and defend against adversarial attacks. We provide extensive empirical evidence on state-of-the-art network architectures to support our theoretical results.
The Mechanism of Empathy and Relationship Commitment Through Emojis: Path to Perspective Taking, Inner Imitation, Emotional Empathy, and Relationship Commitment

We analyzed empathy and relationship commitment mechanisms through emojis. We defined emojis based on theoretical reviews and neuroscientific studies as a mind reading process. Perspective taking, inner imitation, and emoji cognition were independent variables. Emotional empathy and emotional response was the mediating variable. Relationship commitment and behavioral response was the dependent variable. Analyzing the relationship between variables indicated emotional empathy and relationship commitment through emojis having positive (+) relationships with “perspective taking” and “inner imitation,” the two dimensions of mind reading (cognitive and emotional) and, among them, a stronger positive (+) relationship with inner imitation, which is simulation theory’s (ST) cognitive process. Relationship commitment through emojis was strongly related to emotional empathy as a mediating factor than being directly related to cognitive processes (perspective taking, inner imitation). Moreover, considering inner imitation’s influence being greater than perspective taking, relationship commitment through emojis is mainly caused by emotional empathy with inner imitation as a mediating factor.
Stratified and time-aware sampling based adaptive ensemble learning for streaming recommendations

Recommender systems have played an increasingly important role in providing users with tailored suggestions based on their preferences. However, the conventional offline recommender systems cannot handle the ubiquitous data stream well. To address this issue, Streaming Recommender Systems (SRSs) have emerged in recent years, which incrementally train recommendation models on newly received data for effective real-time recommendations. Focusing on new data only benefits addressing concept drift, i.e., the changing user preferences towards items. However, it impedes capturing long-term user preferences. In addition, the commonly existing underload and overload problems should be well tackled for higher accuracy of streaming recommendations. To address these problems, we propose a S tratified and T ime-aware S ampling based A daptive E nsemble L earning framework, called STS-AEL, to improve the accuracy of streaming recommendations. In STS-AEL, we first devise stratified and time-aware sampling to extract representative data from both new data and historical data to address concept drift while capturing long-term user preferences. Also, incorporating the historical data benefits utilizing the idle resources in the underload scenario more effectively. After that, we propose adaptive ensemble learning to efficiently process the overloaded data in parallel with multiple individual recommendation models, and then effectively fuse the results of these models with a sequential adaptive mechanism. Extensive experiments conducted on three real-world datasets demonstrate that STS-AEL, in all the cases, significantly outperforms the state-of-the-art SRSs.
A Soft Computing Approach for Selecting and Combining Spectral Bands

We introduce a soft computing approach for automatically selecting and combining indices from remote sensing multispectral images that can be used for classification tasks. The proposed approach is based on a Genetic-Programming (GP) framework, a technique successfully used in a wide variety of optimization problems. Through GP, it is possible to learn indices that maximize the separability of samples from two different classes. Once the indices specialized for all the pairs of classes are obtained, they are used in pixelwise classification tasks. We used the GP-based solution to evaluate complex classification problems, such as those that are related to the discrimination of vegetation types within and between tropical biomes. Using time series defined in terms of the learned spectral indices, we show that the GP framework leads to superior results than other indices that are used to discriminate and classify tropical biomes.
Learning Domain-Invariant Discriminative Features for Heterogeneous Face Recognition

Heterogeneous face recognition (HFR), referring to matching face images across different domains, is a challenging problem due to the vast cross-domain discrepancy and insufficient pairwise cross-domain training data. This article proposes a quadruplet framework for learning domain-invariant discriminative features (DIDF) for HFR, which integrates domain-level and class-level alignment in one unified network. The domain-level alignment reduces the cross-domain distribution discrepancy. The class-level alignment based on a special quadruplet loss is developed to further diminish the intra-class variations and enlarge the inter-class separability among instances, thus handling the misalignment and adversarial equilibrium problems confronted by the domain-level alignment. With a bidirectional cross-domain data selection strategy, the quadruplet loss-based method prominently enriches the training set and further eliminates the cross-modality shift. Benefiting from the joint supervision and mutual reinforcement of these two components, the domain invariance and class discrimination of identity features are guaranteed. Extensive experiments on the challenging CASIA NIR-VIS 2.0 database, the Oulu-CASIA NIR&VIS database, the BUAA-VisNir database, and the IIIT-D viewed sketch database demonstrate the effectiveness and preferable generalization capability of the proposed method.
Ensemble Distillation for Structured Prediction: Calibrated, Accurate, Fast—Choose Three

Modern neural networks do not always produce well-calibrated predictions, even when trained with a proper scoring function such as cross-entropy.In classification settings, simple methods such as isotonic regression or temperature scaling may be used in conjunction with a held-out dataset to calibrate model outputs.However, extending these methods to structured prediction is not always straightforward or effective; furthermore, a held-out calibration set may not always be available.In this paper, we study ensemble distillation as a general framework for producing wellcalibrated structured prediction models while avoiding the prohibitive inference-time cost of ensembles.We validate this framework on two tasks: named-entity recognition and machine translation.We find that, across both tasks, ensemble distillation produces models which retain much of, and occasionally improve upon, the performance and calibration benefits of ensembles, while only requiring a single model during test-time.
Evaluating Attribution Methods using White-Box LSTMs

Interpretability methods for neural networks are difficult to evaluate because we do not understand the black-box models typically used to test them.This paper proposes a framework in which interpretability methods are evaluated using manually constructed networks, which we call white-box networks, whose behavior is understood a priori.We evaluate five methods for producing attribution heatmaps by applying them to white-box LSTM classifiers for tasks based on formal languages.Although our white-box classifiers solve their tasks perfectly and transparently, we find that all five attribution methods fail to produce the expected model explanations.
The 8-parameter Fisher–Bingham distribution on the sphere

The Fisher–Bingham distribution ( $$\mathrm {FB}_8$$ ) is an eight-parameter family of probability density functions (PDF) on the unit sphere that, under certain conditions, reduce to spherical analogues of bivariate normal PDFs. Due to difficulties in its interpretation and estimation, applications have been mainly restricted to subclasses of $$\mathrm {FB}_8$$ , such as the Kent ( $$\mathrm {FB}_5$$ ) or von Mises–Fisher (vMF) distributions. However, these subclasses often do not adequately describe directional data that are not symmetric along great circles. The normalizing constant of $$\mathrm {FB}_8$$ can be numerically integrated, and recently Kume and Sei showed that it can be computed using an adjusted holonomic gradient method. Both approaches, however, can be computationally expensive. In this paper, I show that the normalization of $$\mathrm {FB}_8$$ can be expressed as an infinite sum consisting of hypergeometric functions, similar to that of the $$\mathrm {FB}_5$$ . This allows the normalization to be computed under summation with adequate stopping conditions. I then fit the $$\mathrm {FB}_8$$ to two datasets using a maximum-likelihood approach and show its improvements over a fit with the more restrictive $$\mathrm {FB}_5$$ distribution.
Research on intelligent substation monitoring by image recognition method

Abstract This study mainly analyzed the improved three-frame difference algorithm for the identification of active targets in the intelligent substation. The improved three-frame difference algorithm introduced the Gaussian mixture background algorithm on the basis of the traditional three-frame difference method. The Gaussian mixture background algorithm, traditional three-frame difference method, and improved three-frame difference method were tested in the actual substation. The results showed that the improved difference method eliminated the non-target background more thoroughly when recognizing the moving target in the image; in the tested video, the improved algorithm had the highest precision and recall ratios for the active target in the video. To sum up, the improved three-frame difference method can more accurately and effectively identify the active targets in the monitoring video, so as to provide an effective support for the unmanned monitoring of intelligent substation.
An Empirical Evaluation of Nuclei Segmentation from H&amp;E Images in a Real Application Scenario

Cell nuclei segmentation is a challenging task, especially in real applications, when the target images significantly differ between them. This task is also challenging for methods based on convolutional neural networks (CNNs), which have recently boosted the performance of cell nuclei segmentation systems. However, when training data are scarce or not representative of deployment scenarios, they may suffer from overfitting to a different extent, and may hardly generalise to images that differ from the ones used for training. In this work, we focus on real-world, challenging application scenarios when no annotated images from a given dataset are available, or when few images (even unlabelled) of the same domain are available to perform domain adaptation. To simulate this scenario, we performed extensive cross-dataset experiments on several CNN-based state-of-the-art cell nuclei segmentation methods. Our results show that some of the existing CNN-based approaches are capable of generalising to target images which resemble the ones used for training. In contrast, their effectiveness considerably degrades when target and source significantly differ in colours and scale.
HUJI-KU at MRP 2020: Two Transition-based Neural Parsers

This paper describes the HUJI-KU system submission to the shared task on Cross-Framework Meaning Representation Parsing (MRP) at the 2020 Conference for Computational Language Learning (CoNLL), employing TUPA and the HIT-SCIR parser, which were, respectively, the baseline system and winning system in the 2019 MRP shared task.Both are transition-based parsers using BERT contextualized embeddings.We generalized TUPA to support the newly-added MRP frameworks and languages, and experimented with multitask learning with the HIT-SCIR parser.We reached 4th place in both the crossframework and cross-lingual tracks.
Super sensitivity and super resolution with quantum teleportation

Abstract We propose a method for quantum enhanced phase estimation based on continuous variable (CV) quantum teleportation. The phase shift probed by a coherent state can be enhanced by repeatedly teleporting the state back to interact with the phase shift again using a supply of two-mode squeezed vacuum states. In this way a sequential protocol exhibiting both super-resolution and super-sensitivity can be obtained due to the coherent addition of the phase shift. The protocol enables Heisenberg-limited sensitivity and super-resolution given sufficiently strong squeezing. The proposed method could be implemented with current or near-term technology of CV teleportation.
A formal framework for deliberated judgment

While the philosophical literature has extensively studied how decisions relate to arguments, reasons and justifications, decision theory almost entirely ignores the latter notions. In this article, we elaborate a formal framework to introduce in decision theory the stance that decision-makers take towards arguments and counter-arguments. We start from a decision situation, where an individual requests decision support. We formally define, as a commendable basis for decision-aid, this individual's deliberated judgment, a notion inspired by Rawls' contributions to the philosophical literature, and embodying the requirement that the decision-maker should carefully examine arguments and counter-arguments. We explain how models of deliberated judgment can be validated empirically. We then identify conditions upon which the existence of a valid model can be taken for granted, and analyze how these conditions can be relaxed. We then explore the significance of our framework for the practice of decision analysis. Our framework opens avenues for future research involving both philosophy and decision theory, as well as empirical implementations.
A wearable device for machine learning based elderly's activity tracking and indoor location system

The number of older people is increasing in many countries. By 2030, it is estimated that 15% of the overall population will be comprised of people aged 65 and above. Hence, the monitoring and tracking of elder activities to ensure they live an active life has become a major research topic in recent years. In this work, an elderly sub-activity tracking system is developed to detect the sub-activity of the elderly based on their physical activities and indoor location. The physical activities tracking system and indoor location system is combined in this project to enhance the context of the elderly activities (i.e. sub-activities as defined in this project). An indoor location system is developed by using Bluetooth Low Energy (BLE) beacon and BLE scanners to measure the Received Signal Strength Indicator (RSSI) signal to detect the location of the elderly. The activity tracking is carried out via a waist wearable device worn by the elderly. Random forest and Support Vector Machine (SVM) are used as machine learning classifiers to predict the activity and indoor location with an accuracy of 95.03% and 86.58%, respectively. The data from activity tracking and indoor location sub-systems will then be combined to derive the sub-activity and push to an online Internet of Things (IoT) platform for remote monitoring and notification.
An analytic theory of shallow networks dynamics for hinge loss classification*

Abstract Neural networks have been shown to perform incredibly well in classification tasks over structured high-dimensional datasets. However, the learning dynamics of such networks is still poorly understood. In this paper we study in detail the training dynamics of a simple type of neural network: a single hidden layer trained to perform a classification task. We show that in a suitable mean-field limit this case maps to a single-node learning problem with a time-dependent dataset determined self-consistently from the average nodes population. We specialize our theory to the prototypical case of a linearly separable data and a linear hinge loss, for which the dynamics can be explicitly solved in the infinite dataset limit. This allows us to address in a simple setting several phenomena appearing in modern networks such as slowing down of training dynamics, crossover between rich and lazy learning, and overfitting. Finally, we assess the limitations of mean-field theory by studying the case of large but finite number of nodes and of training samples.