Reviewerly/exHarmony huggingface dataset:
Fault-Tolerant Connection of Error-Corrected Qubits with Noisy Links

One of the most promising routes towards scalable quantum computing is a modular approach. We show that distinct surface code patches can be connected in a fault-tolerant manner even in the presence of substantial noise along their connecting interface. We quantify analytically and numerically the combined effect of errors across the interface and bulk. We show that the system can tolerate 14 times higher noise at the interface compared to the bulk, with only a small effect on the code's threshold and sub-threshold behavior, reaching threshold with $\sim 1 \%$ bulk errors and $\sim 10 \%$ interface errors. This implies that fault-tolerant scaling of error-corrected modular devices is within reach using existing technology.
Learning to Estimate Shapley Values with Vision Transformers

Transformers have become a default architecture in computer vision, but understanding what drives their predictions remains a challenging problem. Current explanation approaches rely on attention values or input gradients, but these provide a limited view of a model's dependencies. Shapley values offer a theoretically sound alternative, but their computational cost makes them impractical for large, high-dimensional models. In this work, we aim to make Shapley values practical for vision transformers (ViTs). To do so, we first leverage an attention masking approach to evaluate ViTs with partial information, and we then develop a procedure to generate Shapley value explanations via a separate, learned explainer model. Our experiments compare Shapley values to many baseline methods (e.g., attention rollout, GradCAM, LRP), and we find that our approach provides more accurate explanations than existing methods for ViTs.
D'ARTAGNAN: Counterfactual Video Generation

Causally-enabled machine learning frameworks could help clinicians to identify the best course of treatments by answering counterfactual questions. We explore this path for the case of echocardiograms by looking into the variation of the Left Ventricle Ejection Fraction, the most essential clinical metric gained from these examinations. We combine deep neural networks, twin causal networks and generative adversarial methods for the first time to build D'ARTAGNAN (Deep ARtificial Twin-Architecture GeNerAtive Networks), a novel causal generative model. We demonstrate the soundness of our approach on a synthetic dataset before applying it to cardiac ultrasound videos to answer the question: "What would this echocardiogram look like if the patient had a different ejection fraction?". To do so, we generate new ultrasound videos, retaining the video style and anatomy of the original patient, while modifying the Ejection Fraction conditioned on a given input. We achieve an SSIM score of 0.79 and an R2 score of 0.51 on the counterfactual videos. Code and models are available at: https://github.com/HReynaud/dartagnan.
Non-Sequential Decentralized Stochastic Control Revisited: Causality and Static Reducibility

In decentralized stochastic control (or stochastic team theory) and game theory, if there is a pre-defined order in a system in which the agents act, the system is called sequential, otherwise it is non-sequential. Much of the literature on stochastic control theory, such as studies on the existence analysis, approximation methods, and on dynamic programming or other analytical or learning theoretic methods, have focused on sequential systems. The static reduction method for sequential control problems (via change of measures or other techniques), in particular, has been shown to be very effective in arriving at existence, structural, approximation and learning theoretic results. Many practical systems, however, are non-sequential where the order of agents acting is random, and dependent on the realization of solution paths and prior actions taken. The study of such systems is particularly challenging as tools applicable for sequential models are not directly applicable. In this paper, we will study static reducibility of non-sequential stochastic control systems, including by change of measure methods. We revisit the notion of Causality (a definition due to Witsenhausen and which has been refined by Andersland and Tekenetzis), and provide an alternative representation using imaginary agents. Via this representation, we show that Causality, under an absolute continuity condition, allows for an equivalent static model whose reduction is policy-independent. This facilitates much of the stochastic analysis available for sequential systems to also be applicable for non-sequential systems. We further show that under more relaxed conditions on the model, such as solvability, such a reduction, when possible at all, is policy-dependent and thus has limited utility. We will also present a further reduction method for partially nested causal non-sequential systems.
FEATHERS: Federated Architecture and Hyperparameter Search

Deep neural architectures have profound impact on achieved performance in many of today's AI tasks, yet, their design still heavily relies on human prior knowledge and experience. Neural architecture search (NAS) together with hyperparameter optimization (HO) helps to reduce this dependence. However, state of the art NAS and HO rapidly become infeasible with increasing amount of data being stored in a distributed fashion, typically violating data privacy regulations such as GDPR and CCPA. As a remedy, we introduce FEATHERS - $\textbf{FE}$derated $\textbf{A}$rchi$\textbf{T}$ecture and $\textbf{H}$yp$\textbf{ER}$parameter $\textbf{S}$earch, a method that not only optimizes both neural architectures and optimization-related hyperparameters jointly in distributed data settings, but further adheres to data privacy through the use of differential privacy (DP). We show that FEATHERS efficiently optimizes architectural and optimization-related hyperparameters alike, while demonstrating convergence on classification tasks at no detriment to model performance when complying with privacy constraints.
On the construction of quandles of order 3n

We present methods of constructing examples of quandles of order 3n, where n is greater or equal to 3. The necessary and sufficient conditions for the constructed examples to be (i) connected (ii) group (conjugate) (iii) involutory and (iv) Alexander quandles are examined and presented. Two particular examples from these methods are presented for illustration purpose and their properties are obtained, and these are used in classifying the constructed examples up to isomorphism.
Improving Graph-Based Text Representations with Character and Word Level N-grams

Graph-based text representation focuses on how text documents are represented as graphs for exploiting dependency information between tokens and documents within a corpus. Despite the increasing interest in graph representation learning, there is limited research in exploring new ways for graph-based text representation, which is important in downstream natural language processing tasks. In this paper, we first propose a new heterogeneous word-character text graph that combines word and character n-gram nodes together with document nodes, allowing us to better learn dependencies among these entities. Additionally, we propose two new graph-based neural models, WCTextGCN and WCTextGAT, for modeling our proposed text graph. Extensive experiments in text classification and automatic text summarization benchmarks demonstrate that our proposed models consistently outperform competitive baselines and state-of-the-art graph-based models.
Approximate Quantum Algorithms as a Multiphoton Raman Excitation of a Quasicontinuum Edge

Many quantum algorithms can be seen as a transition from a well-defined initial quantum state of a complex quantum system, to an unknown target quantum state, corresponding to a certain eigenvalue either of the Hamiltonian or of a transition operator. Often such a target state corresponds to the minimum energy of a band of states. In this context, approximate quantum calculations imply transition not to the single, minimum energy, state but to a group of states close to the minimum. We consider dynamics and the result of two possible realization of such a process -- transition of population from a single initially populated isolated level to the quantum states at the edge of a band of levels. The first case deals with the time-independent Hamiltonian, while the other with a moving isolated level. We demonstrate that the energy width of the population energy distribution over the band is mainly dictated by the time-energy uncertainty principle, although the specific shape of the distribution depends on the particular setting. We consider the role of the statistics of the coupling matrix elements between the isolated level and the band levels. We have chosen the multiphoton Raman absorption by an ensemble of Rydberg atoms as the model for our analysis, although the results obtained can equally be applied to other quantum computing platforms.
Approximate Reversible Circuits for NISQ-Era Quantum Computers

The synthesis approaches for quantum circuits typically aim at minimizing the number of lines or gates. Given the tight restrictions on those logical resources in physical implementations, we propose to view the problem fundamentally different: Given noisy gates and a fixed number of lines, how can we use them to perform a computation as precisely as possible? In this paper we show approximate circuits can be deployed for computations with limited resources. Performing experiments on a QC simulator, we show that under the influence of noise, approximate circuits can have lower error-rates than exact circuits.
Infinite Recommendation Networks: A Data-Centric Approach

We leverage the Neural Tangent Kernel and its equivalence to training infinitely-wide neural networks to devise $\infty$-AE: an autoencoder with infinitely-wide bottleneck layers. The outcome is a highly expressive yet simplistic recommendation model with a single hyper-parameter and a closed-form solution. Leveraging $\infty$-AE's simplicity, we also develop Distill-CF for synthesizing tiny, high-fidelity data summaries which distill the most important knowledge from the extremely large and sparse user-item interaction matrix for efficient and accurate subsequent data-usage like model training, inference, architecture search, etc. This takes a data-centric approach to recommendation, where we aim to improve the quality of logged user-feedback data for subsequent modeling, independent of the learning algorithm. We particularly utilize the concept of differentiable Gumbel-sampling to handle the inherent data heterogeneity, sparsity, and semi-structuredness, while being scalable to datasets with hundreds of millions of user-item interactions. Both of our proposed approaches significantly outperform their respective state-of-the-art and when used together, we observe 96-105% of $\infty$-AE's performance on the full dataset with as little as 0.1% of the original dataset size, leading us to explore the counter-intuitive question: Is more data what you need for better recommendation?
Characterization of Simplicial Complexes by Counting Simplets Beyond Four Nodes

Simplicial complexes are higher-order combinatorial structures which have been used to represent real-world complex systems. In this paper, we concentrate on the local patterns in simplicial complexes called simplets, a generalization of graphlets. We formulate the problem of counting simplets of a given size in a given simplicial complex. For this problem, we extend a sampling algorithm based on color coding from graphs to simplicial complexes, with essential technical novelty. We theoretically analyze our proposed algorithm named SC3, showing its correctness, unbiasedness, convergence, and time/space complexity. Through the extensive experiments on sixteen real-world datasets, we show the superiority of SC3 in terms of accuracy, speed, and scalability, compared to the baseline methods. Finally, we use the counts given by SC3 for simplicial complex analysis, especially for characterization, which is further used for simplicial complex clustering, where SC3 shows a strong ability of characterization with domain-based similarity.
CodeBERTScore: Evaluating Code Generation with Pretrained Models of Code

Since the rise of neural natural-language-to-code models (NL->Code) that can generate long expressions and statements rather than a single next-token, one of the major problems has been reliably evaluating their generated output. In this paper, we propose CodeBERTScore: an evaluation metric for code generation, which builds on BERTScore (Zhang et al., 2020). Instead of encoding only the generated tokens as in BERTScore, CodeBERTScore also encodes the natural language input preceding the generated code, thus modeling the consistency between the generated code and its given natural language context as well. We perform an extensive evaluation of CodeBERTScore across four programming languages. We find that CodeBERTScore achieves a higher correlation with human preference and with functional correctness than all existing metrics. That is, generated code that receives a higher score by CodeBERTScore is more likely to be preferred by humans, as well as to function correctly when executed. We release five language-specific pretrained models to use with our publicly available code. Our language-specific models have been downloaded more than 1,000,000 times from the Huggingface Hub. Our code and data are available at https://github.com/neulab/code-bert-score
Improved Peel-and-Bound: Methods for Generating Dual Bounds with Multivalued Decision Diagrams

Decision diagrams are an increasingly important tool in cutting-edge solvers for discrete optimization. However, the field of decision diagrams is relatively new, and is still incorporating the library of techniques that conventional solvers have had decades to build. We drew inspiration from the warm-start technique used in conventional solvers to address one of the major challenges faced by decision diagram based methods. Decision diagrams become more useful the wider they are allowed to be, but also become more costly to generate, especially with large numbers of variables. In the original version of this paper, we presented a method of peeling off a sub-graph of previously constructed diagrams and using it as the initial diagram for subsequent iterations that we call peel-and-bound. We tested the method on the sequence ordering problem, and our results indicate that our peel-and-bound scheme generates stronger bounds than a branch-and-bound scheme using the same propagators, and at significantly less computational cost. In this extended version of the paper, we also propose new methods for using relaxed decision diagrams to improve the solutions found using restricted decision diagrams, discuss the heuristic decisions involved with the parallelization of peel-and-bound, and discuss how peel-and-bound can be hyper-optimized for sequencing problems. Furthermore, we test the new methods on the sequence ordering problem and the traveling salesman problem with time-windows (TSPTW), and include an updated and generalized implementation of the algorithm capable of handling any discrete optimization problem. The new results show that peel-and-bound outperforms ddo (a decision diagram based branch-and-bound solver) on the TSPTW. We also close 15 open benchmark instances of the TSPTW.
Achieving acceleration despite very noisy gradients

We present a generalization of Nesterov's accelerated gradient descent algorithm. Our algorithm (AGNES) provably achieves acceleration for smooth convex minimization tasks with noisy gradient estimates if the noise intensity is proportional to the magnitude of the gradient. Nesterov's accelerated gradient descent does not converge under this noise model if the constant of proportionality exceeds one. AGNES fixes this deficiency and provably achieves an accelerated convergence rate no matter how small the signal to noise ratio in the gradient estimate. Empirically, we demonstrate that this is an appropriate model for mini-batch gradients in overparameterized deep learning. Finally, we show that AGNES outperforms stochastic gradient descent with momentum and Nesterov's method in the training of CNNs.
RAFaRe: Learning Robust and Accurate Non-parametric 3D Face Reconstruction from Pseudo 2D&amp;3D Pairs

We propose a robust and accurate non-parametric method for single-view 3D face reconstruction (SVFR). While tremendous efforts have been devoted to parametric SVFR, a visible gap still lies between the result 3D shape and the ground truth. We believe there are two major obstacles: 1) the representation of the parametric model is limited to a certain face database; 2) 2D images and 3D shapes in the fitted datasets are distinctly misaligned. To resolve these issues, a large-scale pseudo 2D\&3D dataset is created by first rendering the detailed 3D faces, then swapping the face in the wild images with the rendered face. These pseudo 2D&3D pairs are created from publicly available datasets which eliminate the gaps between 2D and 3D data while covering diverse appearances, poses, scenes, and illumination. We further propose a non-parametric scheme to learn a well-generalized SVFR model from the created dataset, and the proposed hierarchical signed distance function turns out to be effective in predicting middle-scale and small-scale 3D facial geometry. Our model outperforms previous methods on FaceScape-wild/lab and MICC benchmarks and is well generalized to various appearances, poses, expressions, and in-the-wild environments. The code is released at http://github.com/zhuhao-nju/rafare .
Enhancing Quantum Algorithms for Quadratic Unconstrained Binary Optimization via Integer Programming

To date, research in quantum computation promises potential for outperforming classical heuristics in combinatorial optimization. However, when aiming at provable optimality, one has to rely on classical exact methods like integer programming. State-of-the-art integer programming algorithms can compute strong relaxation bounds even for hard instances, but may have to enumerate a large number of subproblems for determining an optimum solution. If the potential of quantum computing realizes, it can be expected that in particular finding high-quality solutions for hard problems can be done fast. Still, near-future quantum hardware considerably limits the size of treatable problems. In this work, we go one step into integrating the potentials of quantum and classical techniques for combinatorial optimization. We propose a hybrid heuristic for the weighted maximum-cut problem or, equivalently, for quadratic unconstrained binary optimization. The heuristic employs a linear programming relaxation, rendering it well-suited for integration into exact branch-and-cut algorithms. For large instances, we reduce the problem size according to a linear relaxation such that the reduced problem can be handled by quantum machines of limited size. Moreover, we improve the applicability of QAOA, a parameterized quantum algorithm, by deriving optimal parameters for special instances which motivates a parameter estimate for arbitrary instances. We present numerous computational results from real quantum hardware.
Procedural generation of meta-reinforcement learning tasks

Open-endedness stands to benefit from the ability to generate an infinite variety of diverse, challenging environments. One particularly interesting type of challenge is meta-learning ("learning-to-learn"), a hallmark of intelligent behavior. However, the number of meta-learning environments in the literature is limited. Here we describe a parametrized space for simple meta-reinforcement learning (meta-RL) tasks with arbitrary stimuli. The parametrization allows us to randomly generate an arbitrary number of novel simple meta-learning tasks. The parametrization is expressive enough to include many well-known meta-RL tasks, such as bandit problems, the Harlow task, T-mazes, the Daw two-step task and others. Simple extensions allow it to capture tasks based on two-dimensional topological spaces, such as full mazes or find-the-spot domains. We describe a number of randomly generated meta-RL domains of varying complexity and discuss potential issues arising from random generation.
Emotion Detection From Social Media Posts

Over the last few years, social media has evolved into a medium for expressing personal views, emotions, and even business and political proposals, recommendations, and advertisements. We address the topic of identifying emotions from text data obtained from social media posts like Twitter in this research. We have deployed different traditional machine learning techniques such as Support Vector Machines (SVM), Naive Bayes, Decision Trees, and Random Forest, as well as deep neural network models such as LSTM, CNN, GRU, BiLSTM, BiGRU to classify these tweets into four emotion categories (Fear, Anger, Joy, and Sadness). Furthermore, we have constructed a BiLSTM and BiGRU ensemble model. The evaluation result shows that the deep neural network models(BiGRU, to be specific) produce the most promising results compared to traditional machine learning models, with an 87.53 % accuracy rate. The ensemble model performs even better (87.66 %), albeit the difference is not significant. This result will aid in the development of a decision-making tool that visualizes emotional fluctuations.
Differentiable Outlier Detection Enable Robust Deep Multimodal Analysis

Often, deep network models are purely inductive during training and while performing inference on unseen data. Thus, when such models are used for predictions, it is well known that they often fail to capture the semantic information and implicit dependencies that exist among objects (or concepts) on a population level. Moreover, it is still unclear how domain or prior modal knowledge can be specified in a backpropagation friendly manner, especially in large-scale and noisy settings. In this work, we propose an end-to-end vision and language model incorporating explicit knowledge graphs. We also introduce an interactive out-of-distribution (OOD) layer using implicit network operator. The layer is used to filter noise that is brought by external knowledge base. In practice, we apply our model on several vision and language downstream tasks including visual question answering, visual reasoning, and image-text retrieval on different datasets. Our experiments show that it is possible to design models that perform similarly to state-of-art results but with significantly fewer samples and training time.