Miracle-dz/UniFAR-FTUs huggingface dataset:
Explainable Identification of Dementia From Transcripts Using Transformer Networks

Alzheimer’s disease (AD) is the main cause of dementia which is accompanied by loss of memory and may lead to severe consequences in peoples’ everyday life if not diagnosed on time. Very few works have exploited transformer-based networks and despite the high accuracy achieved, little work has been done in terms of model interpretability. In addition, although Mini-Mental State Exam (MMSE) scores are inextricably linked with the identification of dementia, research works face the task of dementia identification and the task of the prediction of MMSE scores as two separate tasks. In order to address these limitations, we employ several transformer-based models, with BERT achieving the highest accuracy accounting for 87.50%. Concurrently, we propose an interpretable method to detect AD patients based on siamese networks reaching accuracy up to 83.75%. Next, we introduce two multi-task learning models, where the main task refers to the identification of dementia (binary classification), while the auxiliary one corresponds to the identification of the severity of dementia (multiclass classification). Our model obtains accuracy equal to 86.25% on the detection of AD patients in the multi-task learning setting. Finally, we present some new methods to identify the linguistic patterns used by AD patients and non-AD ones, including text statistics, vocabulary uniqueness, word usage, correlations via a detailed linguistic analysis, and explainability techniques (LIME). Findings indicate significant differences in language between AD and non-AD patients.
Anomalous Hydrogen Recombination Line Ratios in Ultraluminous Infrared Galaxies

We conducted systematic observations of the H i Brα (4.05 μm) and Brβ (2.63 μm) lines in 52 nearby (z < 0.3) ultraluminous infrared galaxies (ULIRGs) with AKARI. Among 33 ULIRGs wherein the lines are detected, 3 galaxies show anomalous Brβ/Brα line ratios (∼1.0), which are significantly higher than those for case B (0.565). Our observations also show that ULIRGs have a tendency to exhibit higher Brβ/Brα line ratios than those observed in Galactic H ii regions. The high Brβ/Brα line ratios cannot be explained by a combination of dust extinction and case B since dust extinction reduces the ratio. We explore possible causes for the high Brβ/Brα line ratios and show that the observed ratios can be explained by a combination of an optically thick Brα line and an optically thin Brβ line. We simulated the H ii regions in ULIRGs with the Cloudy code, and our results show that the high Brβ/Brα line ratios can be explained by high-density conditions, wherein the Brα line becomes optically thick. To achieve a column density large enough to make the Brα line optically thick within a single H ii region, the gas density must be as high as n ∼ 108 cm−3. We therefore propose an ensemble of H ii regions, in each of which the Brα line is optically thick, to explain the high Brβ/Brα line ratio.
GRiD: GPU-Accelerated Rigid Body Dynamics with Analytical Gradients

We introduce GRiD: a GPU-accelerated library for computing rigid body dynamics with analytical gradients. GRiD was designed to accelerate the nonlinear trajectory opti-mization subproblem used in state-of-the-art robotic planning, control, and machine learning, which requires tens to hundreds of naturally parallel computations of rigid body dynamics and their gradients at each iteration. GRiD leverages URDF parsing and code generation to deliver optimized dynamics kernels that not only expose GPU-friendly computational patterns, but also take advantage of both fine-grained parallelism within each computation and coarse-grained parallelism between computations. Through this approach, when performing multiple computations of rigid body dynamics algorithms, GRiD provides as much as a 7.2x speedup over a state-of-the-art, multi-threaded CPU implementation, and maintains as much as a 2.5x speedup when accounting for I/O overhead. We release GRiD as an open-source library for use by the wider robotics community.
Scalable Average Consensus with Compressed Communications

We propose a new decentralized average consensus algorithm with compressed communication that scales linearly with the network size n. We prove that the proposed method converges to the average of the initial values held locally by the agents of a network when agents are allowed to communicate with compressed messages. The proposed algorithm works for a broad class of compression operators (possibly biased), where agents interact over arbitrary static, undirected, and connected networks. We further present numerical experiments that confirm our theoretical results and illustrate the scalability and communication efficiency of our algorithm.
Image Deraining Convolutional Neural Network For Autonomous Driving

—Perception plays an important role in reliable decision-making for autonomous vehicles. Over the last ten years, huge advances have been made in the field of perception. However, perception in extreme weather conditions is still a difficult problem, especially in rainy weather conditions. In order to improve the detection effect of road targets in rainy environments, we analyze the physical characteristics of the rain layer and propose a deraining convolutional neural network structure. Based on this network structure, we design an ablation experiment and experiment results show that our method can effectively improve the accuracy of object detection in rainy conditions.
Exploring the role of binarity in the origin of the bimodal rotational velocity distribution in stellar clusters

Many young- and intermediate-age massive stellar clusters host bimodal distributions in the rotation rates of their stellar populations, with a dominant peak of rapidly rotating stars and a secondary peak of slow rotators. The origin of this bimodal rotational distribution is currently debated and two main theories have been put forward in the literature. The first is that all/most stars are born as rapid rotators and that interacting binaries break a fraction of the stars, resulting in two populations. The second is that the rotational distribution is a reflection of the early evolution of pre-main sequence stars, in particular, whether they are able to retain or lose their protoplanetary discs during the first few Myr. Here, we test the binary channel by exploiting multi-epoch Very Large Telescope/MUSE observations of NGC 1850, an ∼100 Myr massive cluster in the Large Magellanic Cloud, to search for differences in the binary fractions of the slow- and fast-rotating populations. If binarity is the cause of the rotational bimodality, we would expect that the slowly rotating population should have a much larger binary fraction than the rapid rotators. However, in our data we detect similar fractions of binary stars in the slow and rapidly rotating populations (5.9 ± 1.1 and 4.5 ± 0.6 per cent, respectively). Hence, we conclude that binarity is not a dominant mechanism in the formation of the observed bimodal rotational distributions.
Direct sampling of projected entangled-pair states

Variational Monte Carlo studies employing projected entangled-pair states (PEPS) have recently shown that they can provide answers on long-standing questions such as the nature of the phases in the two-dimensional J 1 − J 2 model. The sampling in these Monte Carlo algorithms is typically performed with Markov Chain Monte Carlo algorithms employing local update rules, which often suﬀer from long autocorrelation times and interdependent samples. We propose a sampling algorithm that generates independent samples from a PEPS, bypassing all problems related to ﬁnite autocorrelation times. This algorithm is a generalization of an existing direct sampling algorithm for unitary tensor networks. We introduce an auxiliary probability distribution from which independent samples can be drawn, and combine it with importance sampling in order to evaluate expectation values accurately. We benchmark our algorithm on the classical Ising model and on variational optimization of two-dimensional quantum spin models.
The Promise of Dataflow Architectures in the Design of Processing Systems for Autonomous Machines

The commercialization of autonomous machines is a thriving sector, and likely to be the next major computing demand driver, after PC, cloud computing, and mobile computing. Nevertheless, a suitable computer architecture for autonomous machines is missing, and many companies are forced to develop ad hoc computing solutions that are neither scalable nor extensible. In this article, we analyze the demands of autonomous machine computing, and argue for the promise of dataﬂow architectures in autonomous machines
Critical Points in Hamiltonian Agnostic Variational Quantum Algorithms

One of the most important properties of classical neural networks is the clustering of local minima of the network near the global minimum, enabling eﬃcient training. This has been observed not only numerically, but also has begun to be analytically understood through the lens of random matrix theory. Inspired by these results in classical machine learning, we show that a certain randomized class of variational quantum algorithms can be mapped to Wishart random ﬁelds on the hypertorus. Then, using the statistical properties of such random processes, we analytically ﬁnd the expected distribution of critical points. Unlike the case for deep neural networks, we show the existence of a transition in the quality of local minima at a number of parameters exponentially large in the problem size. Below this transition, all local minima are concentrated far from the global minimum; above, all local minima are concentrated near the global minimum. This is consistent with previously observed numerical results on the landscape behavior of Hamiltonian agnostic variational quantum algorithms. We give a heuristic explanation as to why ansatzes that depend on the problem Hamiltonian might not suﬀer from these scaling issues. We also verify that our analytic results hold experimentally even at modest system sizes.
FORTAP: Using Formulas for Numerical-Reasoning-Aware Table Pretraining

Tables store rich numerical data, but numerical reasoning over tables is still a challenge. In this paper, we find that the spreadsheet formula, a commonly used language to perform computations on numerical values in spreadsheets, is a valuable supervision for numerical reasoning in tables. Considering large amounts of spreadsheets available on the web, we propose FORTAP, the first exploration to leverage spreadsheet formulas for table pretraining. Two novel self-supervised pretraining objectives are derived from formulas, numerical reference prediction (NRP) and numerical calculation prediction (NCP). While our proposed objectives are generic for encoders, to better capture spreadsheet table layouts and structures, FORTAP is built upon TUTA, the first transformer-based method for spreadsheet table pretraining with tree attention. FORTAP outperforms state-of-the-art methods by large margins on three representative datasets of formula prediction, question answering, and cell type classification, showing the great potential of leveraging formulas for table pretraining.
A Relation-Oriented Clustering Method for Open Relation Extraction

The clustering-based unsupervised relation discovery method has gradually become one of the important methods of open relation extraction (OpenRE). However, high-dimensional vectors can encode complex linguistic information which leads to the problem that the derived clusters cannot explicitly align with the relational semantic classes. In this work, we propose a relation-oriented clustering model and use it to identify the novel relations in the unlabeled data. Specifically, to enable the model to learn to cluster relational data, our method leverages the readily available labeled data of pre-defined relations to learn a relation-oriented representation. We minimize distance between the instance with same relation by gathering the instances towards their corresponding relation centroids to form a cluster structure, so that the learned representation is cluster-friendly. To reduce the clustering bias on predefined classes, we optimize the model by minimizing a joint objective on both labeled and unlabeled data. Experimental results show that our method reduces the error rate by 29.2% and 15.7%, on two datasets respectively, compared with current SOTA methods.
Recursive Hierarchical Projection for Whole-Body Control with Task Priority Transition

Whole-body control (WBC) with task priority transition is an important technology for robots to switch multiple behaviors, change different objectives, and adapt to various environments. Many methods have solved the problem of control continuity in the priority transition process. However, they either increased the computation consumption or sacrificed the accuracy of tasks in practical application. In this work, we propose a Recursive Hierarchical Projection (RHP) matrix and introduce it in Hierarchical Quadratic Programming (HQP). This RHP-HQP scheme can form continuously changing hierarchical projection and regard the WBC problem with task priority transition as a unified formulation. This unified formulation can be smoothly transitioned without increasing computation consumption and solved without losing task accuracy. The comparative simulations of the reactive collision avoidance verify that this priority transition scheme can guarantee high computational efficiency and task accuracy.
Differential equations for the recurrence coefficients of semiclassical orthogonal polynomials and their relation to the Painlevé equations via the geometric approach

In this paper, we present a general scheme for how to relate differential equations for the recurrence coefficients of semiclassical orthogonal polynomials to the Painlevé equations using the geometric framework of the Okamoto Space of Initial Conditions. We demonstrate this procedure in two examples. For semiclassical Laguerre polynomials appearing in Ref. 1, we show how the recurrence coefficients are connected to the fourth Painlevé equation. For discrete orthogonal polynomials associated with the hypergeometric weight appearing in Ref. 2, we discuss the relation of the recurrence coefficients to the sixth Painlevé equation, extending the results of Ref. 3, where a similar approach was used for a discrete system for the same recurrence coefficients. Though the discrete and differential systems here share the same geometry, the construction of the space of initial conditions from the differential system is different and reveals extra considerations that must be made. We also discuss a number of related topics in the context of the geometric approach, such as Hamiltonian forms of the differential equations for the recurrence coefficients, Riccati solutions for special parameter values, and associated discrete Painlevé equations.
Inverse magnetocaloric effect in synthetic antiferromagnets

The magnetocaloric effect in exchange-coupled synthetic-antiferromagnet multilayers is investigated experimentally and theoretically. We observe a temperature-controlled inversion of the effect, where the entropy increases on switching the individual ferromagnetic layers from anti-parallel to parallel alignment near their Curie point. Using a microscopic analytical model as well as numerical atomistic-spin simulations of the system, we explain the observed effect as due to the interplay between the intraand inter-layer exchange interactions, which either add up or counteract to effectively modulate the Curie temperature of the dilute ferromagnetic layers. The proposed method of designing tunable, strongly magneto-caloric materials should be of interest for such applications as heat-assisted spintronics and magnetic refrigeration.
Distribution-free Contextual Dynamic Pricing

Contextual dynamic pricing aims to set personalized prices based on sequential interactions with customers. At each time period, a customer who is interested in purchasing a product comes to the platform. The customer’s valuation for the product is a linear function of contexts, including product and customer features, plus some random market noise. The seller does not observe the customer’s true valuation, but instead needs to learn the valuation by leveraging contextual information and historic binary purchase feedback. Existing models typically assume full or partial knowledge of the random noise distribution. In this paper, we consider contextual dynamic pricing with unknown random noise in the linear valuation model. Our distribution-free pricing policy learns both the contextual function and the market noise simultaneously. A key ingredient of our method is a novel perturbed linear bandit framework, in which a modified linear upper confidence bound algorithm is proposed to balance the exploration of market noise and the exploitation of the current knowledge for better pricing. We establish the regret upper bound and a matching lower bound of our policy in the perturbed linear bandit framework and prove a sublinear regret bound in the considered pricing problem. Finally, we demonstrate the superior performance of our policy on simulations and a real-life auto loan data set. Funding: Y. Liu and W.W. Sun acknowledge support from the National Science Foundation Division of Social and Economic Sciences [Grant NSF-SES 2217440]. Supplemental Material: The supplementary material is available at https://doi.org/10.1287/moor.2023. 1369 .
Exceeding the limits of algorithmic self-calibration in super-resolution imaging

Fourier ptychographic microscopy is a computational imaging technique that provides quantitative phase information and high resolution over a large field-of-view. Although the technique presents numerous advantages over conventional microscopy, model mismatch due to unknown optical aberrations can significantly limit reconstruction quality. Many attempts to address this issue rely on embedding pupil recovery into the reconstruction algorithm. In this paper we demonstrate the limitations of a purely algorithmic approach and evaluate the merits of implementing a simple, dedicated calibration procedure. In simulations, we find that for a target sample reconstruction error, we can image without any aberration corrections up to a maximum aberration magnitude of _/40. When we use algorithmic self-calibration, we can increase the aberration magnitude up to _/10, and with our in situ speckle calibration technique, this working range is extended further to a maximum aberration magnitude of _/3. Hence, one can trade-off complexity for accuracy by using a separate calibration process, which is particularly useful for larger aberrations. © 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Impurity in a heteronuclear two-component Bose mixture

We study the fate of an impurity in an ultracold heteronuclear Bose mixture, focusing on the experimentally relevant case of a 41 K- 87 Rb mixture, with the impurity in a 41 K hyperﬁne state. Our work provides a comprehensive description of an impurity in a BEC mixture with contact interactions across its phase diagram. We present results for the miscible and immiscible regimes, as well as for the impurity in a self-bound quantum droplet. Here, varying the interactions, we ﬁnd novel, exotic states where the impurity localizes either at the center or at the surface of the droplet.
On the Universality of Deep Contextual Language Models

Deep Contextual Language Models (LMs) like ELMO, BERT, and their successors dominate the landscape of Natural Language Processing due to their ability to scale across multiple tasks rapidly by pre-training a single model, followed by task-specific fine-tuning. Furthermore, multilingual versions of such models like XLM-R and mBERT have given promising results in zero-shot cross-lingual transfer, potentially enabling NLP applications in many under-served and under-resourced languages. Due to this initial success, pre-trained models are being used as ‘Universal Language Models’ as the starting point across diverse tasks, domains, and languages. This work explores the notion of ‘Universality’ by identifying seven dimensions across which a universal model should be able to scale, that is, perform equally well or reasonably well, to be useful across diverse settings. We outline the current theoretical and empirical results that support model performance across these dimensions, along with extensions that may help address some of their current limitations. Through this survey, we lay the foundation for understanding the capabilities and limitations of massive contextual language models and help discern research gaps and directions for future work to make these LMs inclusive and fair to diverse applications, users, and linguistic phenomena.
Attention Is Indeed All You Need: Semantically Attention-Guided Decoding for Data-to-Text NLG

Ever since neural models were adopted in data-to-text language generation, they have invariably been reliant on extrinsic components to improve their semantic accuracy, because the models normally do not exhibit the ability to generate text that reliably mentions all of the information provided in the input. In this paper, we propose a novel decoding method that extracts interpretable information from encoder-decoder models’ cross-attention, and uses it to infer which attributes are mentioned in the generated text, which is subsequently used to rescore beam hypotheses. Using this decoding method with T5 and BART, we show on three datasets its ability to dramatically reduce semantic errors in the generated outputs, while maintaining their state-of-the-art quality.
Non-linear Independent Dual System (NIDS) for Discretization-independent Surrogate Modeling over Complex Geometries

Numerical solutions of partial di ﬀ erential equations (PDEs) require expensive simulations, limiting their application in design optimization routines, model-based control, or solution of large-scale inverse problems. Existing Convolutional Neural Network-based frameworks for surrogate modeling require lossy pixelization and data-preprocessing, which is not suitable for realistic engineering applications. Therefore, we propose non-linear independent dual system (NIDS), which is a deep learning surrogate model for discretization-independent, continuous representation of PDE solutions, and can be used for prediction over domains with complex, variable geometries and mesh topologies. NIDS leverages implicit neural representations to develop a non-linear mapping between problem parameters and spatial coordinates to state predictions by combining evaluations of a case-wise parameter network and a point-wise spatial network in a linear output layer. The input features of the spatial network include physical coordinates augmented by a minimum distance function evaluation to implicitly encode the problem geometry. The form of the overall output layer induces a dual system, where each term in the map is non-linear and independent. Further, we propose a minimum distance function-driven weighted sum of NIDS models using a shared parameter network to enforce boundary conditions by construction under certain restrictions. The framework is applied to predict solutions around complex, parametrically-deﬁned geometries on non-parametrically-deﬁned meshes with solutions obtained many orders of magnitude faster than the full order models. Test cases include a vehicle aerodynamics problem with complex geometry and data scarcity, enabled by a training method in which more cases are gradually added as training progresses.