Miracle-dz/UniFAR-FTUs huggingface dataset:
Neural message passing for joint paratope-epitope prediction

Antibodies are proteins in the immune system which bind to antigens to detect and neutralise them. The binding sites in an antibody-antigen interaction are known as the paratope and epitope, respectively, and the prediction of these regions is key to vaccine and synthetic antibody development. Contrary to prior art, we argue that paratope and epitope predictors require asymmetric treatment, and propose distinct neural message passing architectures that are geared towards the specific aspects of paratope and epitope prediction, respectively. We obtain significant improvements on both tasks, setting the new state-of-the-art and recovering favourable qualitative predictions on antigens of relevance to COVID-19.
FedHealth 2: Weighted Federated Transfer Learning via Batch Normalization for Personalized Healthcare

The success of machine learning applications often needs a large quantity of data. Recently, federated learning (FL) is attracting increasing attention due to the demand for data privacy and security, especially in the medical field. However, the performance of existing FL approaches often deteriorates when there exist domain shifts among clients, and few previous works focus on personalization in healthcare. In this article, we propose FedHealth 2, an extension of FedHealth \cite{chen2020fedhealth} to tackle domain shifts and get personalized models for local clients. FedHealth 2 obtains the client similarities via a pretrained model, and then it averages all weighted models with preserving local batch normalization. Wearable activity recognition and COVID-19 auxiliary diagnosis experiments have evaluated that FedHealth 2 can achieve better accuracy (10%+ improvement for activity recognition) and personalized healthcare without compromising privacy and security.
On the Convergence Rate of Off-Policy Policy Optimization Methods with Density-Ratio Correction

In this paper, we study the convergence properties of off-policy policy improvement algorithms with state-action density ratio correction under function approximation setting, where the objective function is formulated as a max-max-min optimization problem. We characterize the bias of the learning objective and present two strategies with finite-time convergence guarantees. In our first strategy, we present algorithm P-SREDA with convergence rate $O(\epsilon^{-3})$, whose dependency on $\epsilon$ is optimal. In our second strategy, we propose a new off-policy actor-critic style algorithm named O-SPIM. We prove that O-SPIM converges to a stationary point with total complexity $O(\epsilon^{-4})$, which matches the convergence rate of some recent actor-critic algorithms in the on-policy setting.
Spectral features of Pb-related color centers in diamond – a systematic photoluminescence characterization

We report on the systematic characterization of the optical properties of diamond color centers based on Pb impurities. An ensemble photoluminescence analysis of their spectral emission was performed at different excitation wavelengths in the 405–520 nm range and at different temperatures in the 4–300 K range. The series of observed spectral features consist of different emission lines associated with Pb-related defects. Finally, a room-temperature investigation of single-photon emitters under 490.5 nm laser excitation is reported, revealing different spectral signatures with respect to those already reported under 514 nm excitation. This work represents a substantial progress with respect to previous studies on Pb-related color centers, both in the attribution of an articulated series of spectral features and in the understanding of the formation process of this type of defect, thus clarifying the potential of this system for high-impact applications in quantum technologies.
Decoherence-free universal quantum computation with symmetric qubit clusters

Christian Boudreault, 2, ∗ Hichem Eleuch, 4, † Michael Hilke, ‡ and Richard MacKenzie § Département des sciences de la nature, Collège militaire royal de Saint-Jean 15 Jacques-Cartier Nord, Saint-Jean-sur-Richelieu, QC, Canada, J3B 8R8 Département de physique, Université de Montréal, Complexe des Sciences, C.P. 6128, succursale Centre-ville, Montréal, QC, Canada, H3C 3J7 Department of Applied Physics and Astronomy, University of Sharjah, Sharjah, United Arab Emirates College of Arts and Sciences, Abu Dhabi University, Abu Dhabi 59911, United Arab Emirates Department of Physics, McGill University, Montréal, QC, Canada, H3A 2T8
Learn to Predict Equilibria via Fixed Point Networks

Systems of competing agents can often be modeled as games. Assuming rationality, the most likely outcomes are given by an equilibrium, e.g. a Nash equilibrium. In many practical settings, games are influenced by context, i.e. additional data beyond the control of any agent (e.g. weather for traffic and fiscal policy for market economies). Often only game equilibria are observed, while the players' true cost functions are unknown. This work introduces Nash Fixed Point Networks (N-FPNs), a class of implicit neural networks that learn to predict the equilibria given only the context. The N-FPN design fuses data-driven modeling with provided constraints on the actions available to agents. N-FPNs are compatible with the recently introduced Jacobian-Free Backpropagation technique for training implicit networks, making them significantly faster to train than prior models. N-FPNs can exploit novel constraint decoupling to avoid costly projections. Provided numerical examples show the efficacy of N-FPNs on atomic and non-atomic games (e.g. traffic routing)
Efficient Passage Retrieval with Hashing for Open-domain Question Answering

Most state-of-the-art open-domain question answering systems use a neural retrieval model to encode passages into continuous vectors and extract them from a knowledge source. However, such retrieval models often require large memory to run because of the massive size of their passage index. In this paper, we introduce Binary Passage Retriever (BPR), a memory-efficient neural retrieval model that integrates a learning-to-hash technique into the state-of-the-art Dense Passage Retriever (DPR) to represent the passage index using compact binary codes rather than continuous vectors. BPR is trained with a multi-task objective over two tasks: efficient candidate generation based on binary codes and accurate reranking based on continuous vectors. Compared with DPR, BPR substantially reduces the memory cost from 65GB to 2GB without a loss of accuracy on two standard open-domain question answering benchmarks: Natural Questions and TriviaQA. Our code and trained models are available at https://github.com/studio-ousia/bpr.
Testing the predictions of axisymmetric distribution functions of galactic dark matter with hydrodynamical simulations

Signal predictions for galactic dark matter (DM) searches often rely on assumptions regarding the DM phase-space distribution function (DF) in halos. This applies to both particle (e.g. p-wave suppressed or Sommerfeld-enhanced annihilation, scattering off atoms, etc.) and macroscopic DM candidates (e.g. microlensing of primordial black holes). As experiments and observations improve in precision, better assessing theoretical uncertainties becomes pressing in the prospect of deriving reliable constraints on DM candidates or trustworthy hints for detection. Most reliable predictions of DFs in halos are based on solving the steady-state collisionless Boltzmann equation (e.g. Eddington-like inversions, action-angle methods, etc.) consistently with observational constraints. One can do so starting from maximal symmetries and a minimal set of degrees of freedom, and then increasing complexity. Key issues are then whether adding complexity, which is computationally costy, improves predictions, and if so where to stop. Clues can be obtained by making predictions for zoomed-in hydrodynamical cosmological simulations in which one can access the true (coarse-grained) phase-space information. Here, we test an axisymmetric extension of the Eddington inversion to predict the full DM DF from its density profile and the total gravitational potential of the system. This permits to go beyond spherical symmetry, and is a priori well suited for spiral galaxies. We show that axisymmetry does not necessarily improve over spherical symmetry because the (observationally unconstrained) angular momentum of the DM halo is not generically aligned with the baryonic one. Theoretical errors are similar to those of the Eddington inversion though, at the 10–20% level for velocity-dependent predictions related to particle DM searches in spiral galaxies. We extensively describe the approach and comment on the results.
SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training

Tabular data underpins numerous high-impact applications of machine learning from fraud detection to genomics and healthcare. Classical approaches to solving tabular problems, such as gradient boosting and random forests, are widely used by practitioners. However, recent deep learning methods have achieved a degree of performance competitive with popular techniques. We devise a hybrid deep learning approach to solving tabular data problems. Our method, SAINT, performs attention over both rows and columns, and it includes an enhanced embedding method. We also study a new contrastive self-supervised pre-training method for use when labels are scarce. SAINT consistently improves performance over previous deep learning methods, and it even outperforms gradient boosting methods, including XGBoost, CatBoost, and LightGBM, on average over a variety of benchmark tasks.
Weight filtrations on Selmer schemes and the effective Chabauty–Kim method

We develop an effective version of the Chabauty–Kim method which gives explicit upper bounds on the number of $S$-integral points on a hyperbolic curve in terms of dimensions of certain Bloch–Kato Selmer groups. Using this, we give a new ‘motivic’ proof that the number of solutions to the $S$-unit equation is bounded uniformly in terms of $\#S$.
Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translation

Knowledge distillation (KD) is commonly used to construct synthetic data for training non-autoregressive translation (NAT) models. However, there exists a discrepancy on low-frequency words between the distilled and the original data, leading to more errors on predicting low-frequency words. To alleviate the problem, we directly expose the raw data into NAT by leveraging pretraining. By analyzing directed alignments, we found that KD makes low-frequency source words aligned with targets more deterministically but fails to align sufficient low-frequency words from target to source. Accordingly, we propose reverse KD to rejuvenate more alignments for low-frequency target words. To make the most of authentic and synthetic data, we combine these complementary approaches as a new training strategy for further boosting NAT performance. We conduct experiments on five translation benchmarks over two advanced architectures. Results demonstrate that the proposed approach can significantly and universally improve translation quality by reducing translation errors on low-frequency words. Encouragingly, our approach achieves 28.2 and 33.9 BLEU points on the WMT14 English-German and WMT16 Romanian-English datasets, respectively. Our code, data, and trained models are available at https://github.com/longyuewangdcu/RLFW-NAT.
Post-mortem on a deep learning contest: a Simpson's paradox and the complementary roles of scale metrics versus shape metrics

To understand better good generalization performance in state-of-the-art neural network (NN) models, and in particular the success of the ALPHAHAT metric based on Heavy-Tailed Self-Regularization (HT-SR) theory, we analyze of a corpus of models that was made publicly-available for a contest to predict the generalization accuracy of NNs. These models include a wide range of qualities and were trained with a range of architectures and regularization hyperparameters. We break ALPHAHAT into its two subcomponent metrics: a scale-based metric; and a shape-based metric. We identify what amounts to a Simpson's paradox: where"scale"metrics (from traditional statistical learning theory) perform well in aggregate, but can perform poorly on subpartitions of the data of a given depth, when regularization hyperparameters are varied; and where"shape"metrics (from HT-SR theory) perform well on each subpartition of the data, when hyperparameters are varied for models of a given depth, but can perform poorly overall when models with varying depths are aggregated. Our results highlight the subtlety of comparing models when both architectures and hyperparameters are varied; the complementary role of implicit scale versus implicit shape parameters in understanding NN model quality; and the need to go beyond one-size-fits-all metrics based on upper bounds from generalization theory to describe the performance of NN models. Our results also clarify further why the ALPHAHAT metric from HT-SR theory works so well at predicting generalization across a broad range of CV and NLP models.
On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study

In adversarial data collection (ADC), a human workforce interacts with a model in real time, attempting to produce examples that elicit incorrect predictions. Researchers hope that models trained on these more challenging datasets will rely less on superficial patterns, and thus be less brittle. However, despite ADC’s intuitive appeal, it remains unclear when training on adversarial datasets produces more robust models. In this paper, we conduct a large-scale controlled study focused on question answering, assigning workers at random to compose questions either (i) adversarially (with a model in the loop); or (ii) in the standard fashion (without a model). Across a variety of models and datasets, we find that models trained on adversarial data usually perform better on other adversarial datasets but worse on a diverse collection of out-of-domain evaluation sets. Finally, we provide a qualitative analysis of adversarial (vs standard) data, identifying key differences and offering guidance for future research.
TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classication

Multiple instance learning (MIL) is a powerful tool to solve the weakly supervised classification in whole slide image (WSI) based pathology diagnosis. However, the current MIL methods are usually based on independent and identical distribution hypothesis, thus neglect the correlation among different instances. To address this problem, we proposed a new framework, called correlated MIL, and provided a proof for convergence. Based on this framework, we devised a Transformer based MIL (TransMIL), which explored both morphological and spatial information. The proposed TransMIL can effectively deal with unbalanced/balanced and binary/multiple classification with great visualization and interpretability. We conducted various experiments for three different computational pathology problems and achieved better performance and faster convergence compared with state-of-the-art methods. The test AUC for the binary tumor classification can be up to 93.09% over CAMELYON16 dataset. And the AUC over the cancer subtypes classification can be up to 96.03% and 98.82% over TCGA-NSCLC dataset and TCGA-RCC dataset, respectively. Implementation is available at: https://github.com/szc19990412/TransMIL.
A Topological Solution of Entanglement for Complex-shaped Parts in Robotic Bin-Picking

This paper addresses the problem of picking up only one object at a time avoiding any entanglement in bin-picking. To cope with a difficult case where the complex-shaped objects are heavily entangled together, we propose a topology-based method that can generate non-tangle grasp positions on a single depth image. The core technique is the entanglement map, which is a feature map to measure the entanglement possibilities obtained from the input image. We use an entanglement map to select probable regions containing graspable objects. The optimum grasping pose is detected from the selected regions considering the collision between robot hand and objects. Experimental results show that our analytic method provides a more comprehensive and intuitive observation of the entanglement and exceeds previous learning-based work [1] in success rates. Especially, our topology-based method does not rely on any object models or time-consuming training process, so that it can be easily adapted to more complex bin-picking scenes.
GROTHENDIECK RINGS OF DEFINABLE SUBASSIGNMENTS AND EQUIVARIANT MOTIVIC MEASURES

This paper studies categories of definable subassignments with some category equivalences to semi-algebraic and constructible subsets of arc spaces of algebraic varieties. These equivalences lead to the identity of certain Grothendieck rings, which allows us to compare the motivic measure of Cluckers-Loeser with that of Denef-Loeser for certain classes of definable subassignments.
Numerical valuation of American basket options via partial differential complementarity problems

We study the principal component analysis based approach introduced by Reisinger and Wittum (2007) and the comonotonic approach considered by Hanbali and Linders (2019) for the approximation of American basket option values via multidimensional partial differential complementarity problems (PDCPs). Both approximation approaches require the solution of just a limited number of low-dimensional PDCPs. It is demonstrated by ample numerical experiments that they define approximations that lie close to each other. Next, an efficient discretisation of the pertinent PDCPs is presented that leads to a favourable convergence behaviour.
Modeling of energetic particle transport in optimized stellarators

Nine stellarator configurations, three quasiaxisymmetric, three quasihelically symmetric and three non-quasisymmetric are scaled to ARIES-CS size and analyzed for energetic particle content. The best performing configurations with regard to energetic particle confinement also perform the best on the neoclassical Γc metric, which attempts to align contours of the second adiabatic invariant with flux surfaces. Quasisymmetric configurations that simultaneously perform well on Γc and quasisymmetry have the best overall confinement, with collisional losses under 3%, approaching the performance of ITER with ferritic inserts.
Statistical optimality conditions for compressive ensembles

We present a framework for the theoretical analysis of ensembles of low-complexity empirical risk minimisers trained on independent random compressions of high-dimensional data. First we introduce a general distribution-dependent upper-bound on the excess risk, framed in terms of a natural notion of compressibility. This bound is independent of the dimension of the original data representation, and explains the in-built regularisation effect of the compressive approach. We then instantiate this general bound to classification and regression tasks, considering Johnson-Lindenstrauss mappings as the compression scheme. For each of these tasks, our strategy is to develop a tight upper bound on the compressibility function, and by doing so we discover distributional conditions of geometric nature under which the compressive algorithm attains minimax-optimal rates up to at most poly-logarithmic factors. In the case of compressive classification, this is achieved with a mild geometric margin condition along with a flexible moment condition that is significantly more general than the assumption of bounded domain. In the case of regression with strongly convex smooth loss functions we find that compressive regression is capable of exploiting spectral decay with near-optimal guarantees. In addition, a key ingredient for our central upper bound is a high probability uniform upper bound on the integrated deviation of dependent empirical processes, which may be of independent interest.
ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Mining

While online conversations can cover a vast amount of information in many different formats, abstractive text summarization has primarily focused on modeling solely news articles. This research gap is due, in part, to the lack of standardized datasets for summarizing online discussions. To address this gap, we design annotation protocols motivated by an issues–viewpoints–assertions framework to crowdsource four new datasets on diverse online conversation forms of news comments, discussion forums, community question answering forums, and email threads. We benchmark state-of-the-art models on our datasets and analyze characteristics associated with the data. To create a comprehensive benchmark, we also evaluate these models on widely-used conversation summarization datasets to establish strong baselines in this domain. Furthermore, we incorporate argument mining through graph construction to directly model the issues, viewpoints, and assertions present in a conversation and filter noisy input, showing comparable or improved results according to automatic and human evaluations.
TSI: Temporal Saliency Integration for Video Action Recognition

Efficient spatiotemporal modeling is an important yet challenging problem for video action recognition. Existing state-of-the-art methods exploit neighboring feature differences to obtain motion clues for short-term temporal modeling with a simple convolution. However, only one local convolution is incapable of handling various kinds of actions because of the limited receptive field. Besides, action-irrelated noises brought by camera movement will also harm the quality of extracted motion features. In this paper, we propose a Temporal Saliency Integration (TSI) block, which mainly contains a Salient Motion Excitation (SME) module and a Cross-perception Temporal Integration (CTI) module. Specifically, SME aims to highlight the motion-sensitive area through spatial-level local-global motion modeling, where the saliency alignment and pyramidal motion modeling are conducted successively between adjacent frames to capture motion dynamics with fewer noises caused by misaligned background. CTI is designed to perform multi-perception temporal modeling through a group of separate 1D convolutions respectively. Meanwhile, temporal interactions across different perceptions are integrated with the attention mechanism. Through these two modules, long short-term temporal relationships can be encoded efficiently by introducing limited additional parameters. Extensive experiments are conducted on several popular benchmarks (i.e., Something-Something V1&V2, Kinetics-400, UCF-101, and HMDB-51), which demonstrate the effectiveness of our proposed method.
Photon decay in ultrahigh-energy air showers: Stringent bound on Lorentz violation

In extensive air showers induced by ultra-high energy (UHE) cosmic rays, secondary photons are expected to be produced at energies far above those accessible by other means. It has been shown that the decay of such photons, as possible in certain theories allowing the violation of Lorentz invariance, can lead to significant changes of the shower development. Based on observations of the average depth of shower maximum 〈Xmax〉, a stringent bound on the violation of Lorentz invariance has been placed in a previous work. In this contribution we include the shower-toshower fluctuations σ(Xmax) as an additional observable. The combined comparison of 〈Xmax〉 and σ(Xmax) to shower observations allows a much stricter test of the possible decay of UHE photons, improving the existing bound by a factor 50.