Reviewerly/exHarmony huggingface dataset:
On Evaluating the Generalization of LSTM Models in Formal Languages

Recurrent Neural Networks (RNNs) are theoretically Turing-complete and established themselves as a dominant model for language processing. Yet, there still remains an uncertainty regarding their language learning capabilities. In this paper, we empirically evaluate the inductive learning capabilities of Long Short-Term Memory networks, a popular extension of simple RNNs, to learn simple formal languages, in particular $a^nb^n$, $a^nb^nc^n$, and $a^nb^nc^nd^n$. We investigate the influence of various aspects of learning, such as training data regimes and model capacity, on the generalization to unobserved samples. We find striking differences in model performances under different training settings and highlight the need for careful analysis and assessment when making claims about the learning capabilities of neural network models.
Homologous Component Analysis for Domain Adaptation

Covariate shift assumption-based domain adaptation approaches usually utilize only one common transformation to align marginal distributions and make conditional distributions preserved. However, one common transformation may cause loss of useful information, such as variances and neighborhood relationship in both source and target domains. To address this problem, we propose a novel method called homologous component analysis (HCA) where we try to find two totally different but homologous transformations to align distributions with side information and make conditional distributions preserved. As it is hard to find a closed-form solution to the corresponding optimization problem, we solve them by means of the alternating direction minimizing method (ADMM) in the context of Stiefel manifolds. We also provide a generalization error bound for domain adaptation in the semi-supervised case, and two transformations can help to decrease this upper bound more than only one common transformation does. Extensive experiments on synthetic and real data show the effectiveness of the proposed method by comparing its classification accuracy with the state-of-the-art methods, and the numerical evidence on chordal distance and Frobenius distance shows that resulting optimal transformations are different.
Worst-Case Regret Bounds for Exploration via Randomized Value Functions

This paper studies a recent proposal to use randomized value functions to drive exploration in reinforcement learning. These randomized value functions are generated by injecting random noise into the training data, making the approach compatible with many popular methods for estimating parameterized value functions. By providing a worst-case regret bound for tabular finite-horizon Markov decision processes, we show that planning with respect to these randomized value functions can induce provably efficient exploration.
Democratising online content moderation: A constitutional framework

Freedom of expression is one of the cornerstones on which democracy is based. This non-exhaustive statement firmly clashes with the troubling evolution of the algorithmic society where artificial intelligence technologies govern the flow of information online according to opaque technical standards established by social media platforms. These actors are usually neither accountable nor responsible for contents uploaded or generated by the users. Nevertheless, online content moderation affects users’ fundamental rights and democratic values, especially since online platforms autonomously set standards for content removal on a global scale. Despite their crucial role in governing the flow of information online, social media platforms are not required to ensure transparency and explanation of their decision-making processes. Within this framework, this work aims to show how the liberal paradigm of protection of the right to free speech is no longer enough to protect democratic values in the digital environment, since the flow of information is actively organised by business interests, driven by profit-maximisation rather than democracy, transparency or accountability. The role of free speech is still paramount. However, the challenges raised by the algorithmic society leads to focus on enhancing the positive dimension of this fundamental right by introducing new users’ rights and transparency and accountability obligations for social media to inject democratic values in the digital environment.
Theory of robust multiqubit nonadiabatic gates for trapped ions

The prevalent approach to executing quantum algorithms on quantum computers is to break-down the algorithms to a concatenation of universal gates, typically single and two-qubit gates. However such a decomposition results in long gate sequences which are exponential in the qubit register size. Furthermore, gate fidelities tend to decrease when acting in larger qubit registers. Thus high-fidelity implementations in large qubit registers is still a prominent challenge. Here we propose and investigate multi-qubit entangling gates for trapped-ions. Our gates couple many qubits at once, allowing to decrease the total number of gates used while retaining a high gate fidelity. Our method employs all of the normal-modes of motion of the ion chain, which allows to operate outside of the adiabatic regime and at rates comparable to the secular ion-trapping frequency. Furthermore we extend our method for generating Hamiltonians which are suitable for quantum analog simulations, such as a nearest-neighbour spin Hamiltonian or the Su-Schrieffer-Heeger Hamiltonian.
Optimal design strategy for non-Abelian geometric phases using Abelian gauge fields based on quantum metric

A photonic implementation of a SU(2) holonomic quantum gate, i.e. a single-qubit gate, is realized via a tripod-like structure of four coupled waveguides and characterized by its invariant Wilson loop. The gate operates on the degenerate dark subspace of the structure. The properties of the dark subspace are determined by the quantum metric, which provides an optimization scheme for the structure's design, thereby ensuring the required adiabaticity and providing a useful error measure.
Emotion Analysis From Turkish Tweets Using Deep Neural Networks

Text data analysis of social media is becoming more and more important since it includes the most recent information on what people think about. Likewise, emotion is one of the most valuable parts of human communication, emotion analysis is a type of information extraction process which identifies the emotional states of a given text. In this study, we investigated the performance of deep neural networks on emotion analysis from Turkish tweets. For this, we examined three different deep learning architectures including artificial neural network (ANN), convolutional neural network (CNN) and recurrent neural network (RNN) with long short-term memory (LSTM). Besides, we curated a dataset of Turkish tweets and annotated each tweet automatically for six emotion categories using a lexicon-based approach. For the evaluation, we conducted a set of experiments for each architecture. The results showed that the lexicon-based automatic annotation of tweets is valid. Secondly, ANN produced the worst result as expected, and CNN resulted in the highest score of 0.74 in terms of accuracy measure. Experiments also showed that our proposed approach for emotion analysis of tweets in Turkish performs better than state-of-the-art in this topic.
Extreme learning with chemical reaction optimization for stock volatility prediction

Abstract Extreme learning machine (ELM) allows for fast learning and better generalization performance than conventional gradient-based learning. However, the possible inclusion of non-optimal weight and bias due to random selection and the need for more hidden neurons adversely influence network usability. Further, choosing the optimal number of hidden nodes for a network usually requires intensive human intervention, which may lead to an ill-conditioned situation. In this context, chemical reaction optimization (CRO) is a meta-heuristic paradigm with increased success in a large number of application areas. It is characterized by faster convergence capability and requires fewer tunable parameters. This study develops a learning framework combining the advantages of ELM and CRO, called extreme learning with chemical reaction optimization (ELCRO). ELCRO simultaneously optimizes the weight and bias vector and number of hidden neurons of a single layer feed-forward neural network without compromising prediction accuracy. We evaluate its performance by predicting the daily volatility and closing prices of BSE indices. Additionally, its performance is compared with three other similarly developed models—ELM based on particle swarm optimization, genetic algorithm, and gradient descent—and find the performance of the proposed algorithm superior. Wilcoxon signed-rank and Diebold–Mariano tests are then conducted to verify the statistical significance of the proposed model. Hence, this model can be used as a promising tool for financial forecasting.
Artificial intelligence and computer simulation models in critical illness

Widespread implementation of electronic health records has led to the increased use of artificial intelligence (AI) and computer modeling in clinical medicine. The early recognition and treatment of critical illness are central to good outcomes but are made difficult by, among other things, the complexity of the environment and the often non-specific nature of the clinical presentation. Increasingly, AI applications are being proposed as decision supports for busy or distracted clinicians, to address this challenge. Data driven "associative" AI models are built from retrospective data registries with missing data and imprecise timing. Associative AI models lack transparency, often ignore causal mechanisms, and, while potentially useful in improved prognostication, have thus far had limited clinical applicability. To be clinically useful, AI tools need to provide bedside clinicians with actionable knowledge. Explicitly addressing causal mechanisms not only increases validity and replicability of the model, but also adds transparency and helps gain trust from the bedside clinicians for real world use of AI models in teaching and patient care.
Lessons Learned from Designing an AI-Enabled Diagnosis Tool for Pathologists

Despite the promises of data-driven artificial intelligence (AI), little is known about how we can bridge the gulf between traditional physician-driven diagnosis and a plausible future of medicine automated by AI. Specifically, how can we involve AI usefully in physicians' diagnosis workflow given that most AI is still nascent and error-prone (e.g., in digital pathology)? To explore this question, we first propose a series of collaborative techniques to engage human pathologists with AI given AI's capabilities and limitations, based on which we prototype Impetus - a tool where an AI takes various degrees of initiatives to provide various forms of assistance to a pathologist in detecting tumors from histological slides. We summarize observations and lessons learned from a study with eight pathologists and discuss recommendations for future work on human-centered medical AI systems.
Benchmarking near-term devices with quantum error correction

Now that ever more sophisticated devices for quantum computing are being developed, we require ever more sophisticated benchmarks. This includes a need to determine how well these devices support the techniques required for quantum error correction. In this paper we introduce the \texttt{topological\_codes} module of Qiskit-Ignis, which is designed to provide the tools necessary to perform such tests. Specifically, we use the \texttt{RepetitionCode} and \texttt{GraphDecoder} classes to run tests based on the repetition code and process the results. As an example, data from a 43 qubit code running on IBM's \emph{Rochester} device is presented.
Deep generative models for galaxy image simulations

Image simulations are essential tools for preparing and validating the analysis of current and future wide-field optical surveys. However, the galaxy models used as the basis for these simulations are typically limited to simple parametric light profiles, or use a fairly limited amount of available space-based data. In this work, we propose a methodology based on Deep Generative Models to create complex models of galaxy morphologies that may meet the image simulation needs of upcoming surveys. We address the technical challenges associated with learning this morphology model from noisy and PSF-convolved images by building a hybrid Deep Learning/physical Bayesian hierarchical model for observed images, explicitly accounting for the Point Spread Function and noise properties. The generative model is further made conditional on physical galaxy parameters, to allow for sampling new light profiles from specific galaxy populations. We demonstrate our ability to train and sample from such a model on galaxy postage stamps from the HST/ACS COSMOS survey, and validate the quality of the model using a range of second- and higher-order morphology statistics. Using this set of statistics, we demonstrate significantly more realistic morphologies using these deep generative models compared to conventional parametric models. To help make these generative models practical tools for the community, we introduce GalSim-Hub, a community-driven repository of generative models, and a framework for incorporating generative models within the GalSim image simulation software.
Object Discovery From a Single Unlabeled Image by Mining Frequent Itemsets With Multi-Scale Features

The goal of our work is to discover dominant objects in a very general setting where only a single unlabeled image is given. This is far more challenge than typical co-localization or weakly-supervised localization tasks. To tackle this problem, we propose a simple but effective pattern mining-based method, called Object Location Mining (OLM), which exploits the advantages of data mining and feature representation of pre-trained convolutional neural networks (CNNs). Specifically, we first convert the feature maps from a pre-trained CNN model into a set of transactions, and then discovers frequent patterns from transaction database through pattern mining techniques. We observe that those discovered patterns, i.e., co-occurrence highlighted regions, typically hold appearance and spatial consistency. Motivated by this observation, we can easily discover and localize possible objects by merging relevant meaningful patterns. Extensive experiments on a variety of benchmarks demonstrate that OLM achieves competitive localization performance compared with the state-of-the-art methods. We also evaluate our approach compared with unsupervised saliency detection methods and achieves competitive results on seven benchmark datasets. Moreover, we conduct experiments on fine-grained classification to show that our proposed method can locate the entire object and parts accurately, which can benefit to improving the classification results significantly.
Benchmarking Orthogroup Inference Accuracy: Revisiting Orthobench

Abstract Orthobench is the standard benchmark to assess the accuracy of orthogroup inference methods. It contains 70 expert-curated reference orthogroups (RefOGs) that span the Bilateria and cover a range of different challenges for orthogroup inference. Here, we leveraged improvements in tree inference algorithms and computational resources to reinterrogate these RefOGs and carry out an extensive phylogenetic delineation of their composition. This phylogenetic revision altered the membership of 31 of the 70 RefOGs, with 24 subject to extensive revision and 7 that required minor changes. We further used these revised and updated RefOGs to provide an assessment of the orthogroup inference accuracy of widely used orthogroup inference methods. Finally, we provide an open-source benchmarking suite to support the future development and use of the Orthobench benchmark.
Limitations of error corrected quantum annealing in improving the performance of Boltzmann machines

Boltzmann machines are the basis of several deep learning methods that have been successfully applied to both supervised and unsupervised machine learning tasks. These models assume that a dataset is generated according to a Boltzmann distribution, and the goal of the training procedure is to learn the set of parameters that most closely match the input data distribution. Training such models is difficult due to the intractability of traditional sampling techniques, and proposals using quantum annealers for sampling hope to mitigate the sampling cost. However, real physical devices will inevitably be coupled to the environment, and the strength of this coupling affects the effective temperature of the distributions from which a quantum annealer samples. To counteract this problem, error correction schemes that can effectively reduce the temperature are needed if there is to be some benefit in using quantum annealing for problems at a larger scale, where we might expect the effective temperature of the device to not be sufficiently low. To this end, we have applied nested quantum annealing correction (NQAC) to do unsupervised learning with a small bars and stripes dataset, and to do supervised learning with a coarse-grained MNIST dataset. For both datasets we demonstrate improved training and a concomitant effective temperature reduction at higher noise levels relative to the unencoded case. We also find better performance overall with longer anneal times and offer an interpretation of the results based on a comparison to simulated quantum annealing and spin vector Monte Carlo. A counterintuitive aspect of our results is that the output distribution generally becomes less Gibbs-like with increasing nesting level and increasing anneal times, which shows that improved training performance can be achieved without equilibration to the target Gibbs distribution.
High-Efficiency Measurement of an Artificial Atom Embedded in a Parametric Amplifier

A crucial limit to measurement efficiencies of superconducting circuits comes from losses involved when coupling to an external quantum amplifier. Here, we realize a device circumventing this problem by directly embedding a two-level artificial atom, comprised of a transmon qubit, within a flux-pumped Josephson parametric amplifier. Surprisingly, this configuration is able to enhance dispersive measurement without exposing the qubit to appreciable excess backaction. This is accomplished by engineering the circuit to permit high-power operation that reduces information loss to unmonitored channels associated with the amplification and squeezing of quantum noise. By mitigating the effects of off-chip losses downstream, the on-chip gain of this device produces end-to-end measurement efficiencies of up to 80 percent. Our theoretical model accurately describes the observed interplay of gain and measurement backaction, and delineates the parameter space for future improvement. The device is compatible with standard fabrication and measurement techniques, and thus provides a route for definitive investigations of fundamental quantum effects and quantum control protocols.
An Adaptive Surrogate Modeling Based on Deep Neural Networks for Large-Scale Bayesian Inverse Problems

In Bayesian inverse problems, surrogate models are often constructed to speed up the computational procedure, as the parameter-to-data map can be very expensive to evaluate. However, due to the curse of dimensionality and the nonlinear concentration of the posterior, traditional surrogate approaches (such us the polynomial-based surrogates) are still not feasible for large scale problems. To this end, we present in this work an adaptive multi-fidelity surrogate modeling framework based on deep neural networks (DNNs), motivated by the facts that the DNNs can potentially handle functions with limited regularity and are powerful tools for high dimensional approximations. More precisely, we first construct offline a DNNs-based surrogate according to the prior distribution, and then, this prior-based DNN-surrogate will be adaptively \& locally refined online using only a few high-fidelity simulations. In particular, in the refine procedure, we construct a new shallow neural network that view the previous constructed surrogate as an input variable -- yielding a composite multi-fidelity neural network approach. This makes the online computational procedure rather efficient. Numerical examples are presented to confirm that the proposed approach can obtain accurate posterior information with a limited number of forward simulations.
Parsing All: Syntax and Semantics, Dependencies and Spans

Both syntactic and semantic structures are key linguistic contextual clues, in which parsing the latter has been well shown beneficial from parsing the former.However, few works ever made an attempt to let semantic parsing help syntactic parsing.As linguistic representation formalisms, both syntax and semantics may be represented in either span (constituent/phrase) or dependency, on both of which joint learning was also seldom explored.In this paper, we propose a novel joint model of syntactic and semantic parsing on both span and dependency representations, which incorporates syntactic information effectively in the encoder of neural network and benefits from two representation formalisms in a uniform way.The experiments show that semantics and syntax can benefit each other by optimizing joint objectives.Our single model achieves new state-of-the-art or competitive results on both span and dependency semantic parsing on Propbank benchmarks and both dependency and constituent syntactic parsing on Penn Treebank.
Merged-Element Transmon

Transmon qubits are ubiquitous in the pursuit of quantum computing using superconducting circuits. However, they have some drawbacks that still need to be addressed. Most importantly, the scalability of transmons is limited by the large device footprint needed to reduce the participation of the lossy capacitive parts of the circuit. In this work, we investigate and evaluate losses in an alternative device geometry, namely, the merged-element transmon (mergemon). To this end, we replace the large external shunt capacitor of a traditional transmon with the intrinsic capacitance of a Josephson junction and achieve an approximately 100 times reduction in qubit dimensions. We report the implementation of the mergemon using a sputtered $\mathrm{Nb}$--amorphous-$\mathrm{Si}$--$\mathrm{Nb}$ trilayer film. In an experiment below 10 mK, the frequency of the readout resonator, capacitively coupled to the mergemon, exhibits a qubit-state-dependent shift in the low-power regime. The device also demonstrates the single-photon and multiphoton transitions that represent a weakly anharmonic system in two-tone spectroscopy. The transition spectra are explained well with master-equation simulations. A participation-ratio analysis identifies the dielectric loss of the amorphous-$\mathrm{Si}$ tunnel barrier and its interfaces as the dominant source for qubit relaxation. We expect the mergemon to achieve high coherence for relatively small device dimensions when implemented with a low-loss, epitaxially grown, and lattice-matched trilayer.