Miracle-dz/UniFAR-FTUs huggingface dataset:
Document Embedding for Scientific Articles: Efficacy of Word Embeddings vs TFIDF

Over the last few years, neural network derived word embeddings became popular in the natural language processing literature. Studies conducted have mostly focused on the quality and application of word embeddings trained on public available corpuses such as Wikipedia or other news and social media sources. However, these studies are limited to generic text and thus lack technical and scientific nuances such as domain specific vocabulary, abbreviations, or scientific formulas which are commonly used in academic context. This research focuses on the performance of word embeddings applied to a large scale academic corpus. More specifically, we compare quality and efficiency of trained word embeddings to TFIDF representations in modeling content of scientific articles. We use a word2vec skip-gram model trained on titles and abstracts of about 70 million scientific articles. Furthermore, we have developed a benchmark to evaluate content models in a scientific context. The benchmark is based on a categorization task that matches articles to journals for about 1.3 million articles published in 2017. Our results show that content models based on word embeddings are better for titles (short text) while TFIDF works better for abstracts (longer text). However, the slight improvement of TFIDF for larger text comes at the expense of 3.7 times more memory requirement as well as up to 184 times higher computation times which may make it inefficient for online applications. In addition, we have created a 2-dimensional visualization of the journals modeled via embeddings to qualitatively inspect embedding model. This graph shows useful insights and can be used to find competitive journals or gaps to propose new journals.
Strategyproof Mechanisms For Group-Fair Facility Location Problems

We study the facility location problems where agents are located on a real line and divided into groups based on criteria such as ethnicity or age. Our aim is to design mechanisms to locate a facility to approximately minimize the costs of groups of agents to the facility fairly while eliciting the agents' locations truthfully. We first explore various well-motivated group fairness cost objectives for the problems and show that many natural objectives have an unbounded approximation ratio. We then consider minimizing the maximum total group cost and minimizing the average group cost objectives. For these objectives, we show that existing classical mechanisms (e.g., median) and new group-based mechanisms provide bounded approximation ratios, where the group-based mechanisms can achieve better ratios. We also provide lower bounds for both objectives. To measure fairness between groups and within each group, we study a new notion of intergroup and intragroup fairness (IIF) . We consider two IIF objectives and provide mechanisms with tight approximation ratios.
An algorithm for Egyptian fraction representations with restricted denominators

A unit fraction representation of a rational number r is a finite sum of reciprocals of positive integers that equals r. Of particular interest is the case when all denominators in the representation are distinct, resulting in an Egyptian fraction representation of r. Common algorithms for computing Egyptian fraction representations of a given rational number tend to result in extremely large denominators and cannot be adapted to restrictions on the allowed denominators. We describe an algorithm for finding all unit fraction representations of a given rational number using denominators from a given finite multiset of positive integers. The freely available algorithm, implemented in Scheme, is particularly well suited to computing dense Egyptian fraction representations, where the allowed denominators have a prescribed maximum.
A Multi-Lead Fusion Method for the Accurate Delineation of QRS Complex Location in 12 Lead ECG Signal

This paper presents a multi-lead fusion method for the accurate and automated detection of the QRS complex location in 12 lead ECG (Electrocardiogram) signals. The proposed multi-lead fusion method accurately delineates the QRS complex by the fusion of detected QRS complexes of the individual 12 leads. The proposed algorithm consists of two major stages. Firstly, the QRS complex location of each lead is detected by the single lead QRS detection algorithm. Secondly, the multi-lead fusion algorithm combines the information of the QRS complex locations obtained in each of the 12 leads. The performance of the proposed algorithm is improved in terms of Sensitivity and Positive Predictivity by discarding the false positives. The proposed method is validated on the ECG signals with various artifacts, inter and intra subject variations. The performance of the proposed method is validated on the long duration recorded ECG signals of St. Petersburg INCART database with Sensitivity of 99.87% and Positive Predictivity of 99.96% and on the short duration recorded ECG signals of CSE (Common Standards for Electrocardiography) multi-lead database with 100% Sensitivity and 99.13% Positive Predictivity.
Nonparametric Regression with Shallow Overparameterized Neural Networks Trained by GD with Early Stopping

We explore the ability of overparameterized shallow neural networks to learn Lipschitz regression functions with and without label noise when trained by Gradient Descent (GD). To avoid the problem that in the presence of noisy labels, neural networks trained to nearly zero training error are inconsistent on this class, we propose an early stopping rule that allows us to show optimal rates. This provides an alternative to the result of Hu et al. (2021) who studied the performance of $\ell 2$ -regularized GD for training shallow networks in nonparametric regression which fully relied on the infinite-width network (Neural Tangent Kernel (NTK)) approximation. Here we present a simpler analysis which is based on a partitioning argument of the input space (as in the case of 1-nearest-neighbor rule) coupled with the fact that trained neural networks are smooth with respect to their inputs when trained by GD. In the noise-free case the proof does not rely on any kernelization and can be regarded as a finite-width result. In the case of label noise, by slightly modifying the proof, the noise is controlled using a technique of Yao, Rosasco, and Caponnetto (2007).
Metalearning Linear Bandits by Prior Update

Fully Bayesian approaches to sequential decision-making assume that problem parameters are generated from a known prior. In practice, such information is often lack-ing. This problem is exacerbated in setups with partial information, where a misspeciﬁed prior may lead to poor exploration and performance. In this work we prove, in the context of stochastic linear bandits and Gaussian priors, that as long as the prior is suﬃciently close to the true prior, the performance of the applied algorithm is close to that of the algorithm that uses the true prior. Furthermore, we address the task of learning the prior through metalearning, where a learner updates her estimate of the prior across multiple task instances in order to improve performance on future tasks. We provide an algorithm and regret bounds, demonstrate its eﬀectiveness in comparison to an algorithm that knows the correct prior, and support our theoretical results empirically. Our theoretical results hold for a broad class of algorithms, including Thompson Sampling and Information Directed Sampling.
Mechanism of exotic density-wave and beyond-Migdal unconventional superconductivity in kagome metal AV3Sb5 (A = K, Rb, Cs)

Exotic quantum phase transitions in metals, such as the electronic nematic state, have been discovered one after another and found to be universal now. The emergence of unconventional density-wave (DW) order in frustrated kagome metal AV3Sb5 and its interplay with exotic superconductivity attract increasing attention. We find that the DW in kagome metal is the bond order, because the sizable intersite attraction is caused by the quantum interference among paramagnons. This mechanism is important in kagome metals because the geometrical frustration prohibits the freezing of paramagnons. In addition, we uncover that moderate bond-order fluctuations mediate sizable pairing glue, and this mechanism gives rise to both singlet s-wave and triplet p-wave superconductivity. Furthermore, characteristic pressure-induced phase transitions in CsV3Sb5 are naturally understood by the present theory. Thus, both the exotic density wave and the superconductivity in geometrically frustrated kagome metals are explained by the quantum interference mechanism.
Analysis of the superdeterministic Invariant-set theory in a hidden-variable setting

A recent proposal for a superdeterministic account of quantum mechanics, named Invariant-set theory, appears to bring ideas from several diverse fields like chaos theory, number theory and dynamical systems to quantum foundations. However, a clear cut hidden-variable model has not been developed, which makes it difficult to assess the Proposal from a quantum foundational perspective. In this article, we first build a hidden-variable model based on the Proposal, and then critically analyse several aspects of the Proposal using the model. We show that several arguments related to counter-factual measurements, nonlocality, non-commutativity of quantum observables, measurement independence etcetera that appear to work in the Proposal fail when considered in our model. We further show that our model is not only superdeterministic but also nonlocal, with an ontic quantum state. We argue that the bit string defined in the model is a hidden variable and that it contains redundant information. Lastly, we apply the analysis developed in a previous work (Sen I, Valentini A, 2020 Proc. R. Soc. A, 476, 20200214) to illustrate the issue of superdeterministic conspiracy in the model. Our results lend further support to the view that superdeterminism is unlikely to solve the puzzle posed by the Bell correlations.
Cautious Actor-Critic

The oscillating performance of off-policy learning and persisting errors in the actor-critic (AC) setting call for algorithms that can conservatively learn to suit the stability-critical applications better. In this paper, we propose a novel off-policy AC algorithm cautious actor-critic (CAC). The name cautious comes from the doubly conservative nature that we exploit the classic policy interpolation from conservative policy iteration for the actor and the entropy-regularization of conservative value iteration for the critic. Our key observation is the entropy-regularized critic facilitates and simplifies the unwieldy interpolated actor update while still ensuring robust policy improvement. We compare CAC to state-of-the-art AC methods on a set of challenging continuous control problems and demonstrate that CAC achieves comparable performance while significantly stabilizes learning.
Attitude Reconstruction From Inertial Measurement: Mitigating Runge Effect for Dynamic Applications

Time-equispaced inertial measurements are practically used as inputs for motion determination. Polynomial interpolation is a common technique of recovering the gyroscope signal but is subject to a fundamentally numerical stability problem due to the Runge effect on equispaced samples. This article reviews the theoretical results of Runge phenomenon in related areas and proposes a straightforward borrowing-and-cutting (BAC) strategy to depress it. It employs the neighboring samples for higher order polynomial interpolation but only uses the middle polynomial segment in the actual time interval. The BAC strategy has been incorporated into attitude computation by functional iteration, leading to accuracy benefit of several orders of magnitude under the classical coning motion. It would potentially bring significant benefits to the inertial navigation computation under sustained dynamic motions.
Stability for the Calderón’s problem for a class of anisotropic conductivities via an ad hoc misfit functional

We address the stability issue in Calderón’s problem for a special class of anisotropic conductivities of the form σ = γA in a Lipschitz domain Ω⊂Rn , n ⩾ 3, where A is a known Lipschitz continuous matrix-valued function and γ is the unknown piecewise affine scalar function on a given partition of Ω. We define an ad hoc misfit functional encoding our data and establish stability estimates for this class of anisotropic conductivity in terms of both the misfit functional and the more commonly used local Dirichlet-to-Neumann map.
Remote Sensing Image Change Detection With Transformers

Modern change detection (CD) has achieved remarkable success by the powerful discriminative ability of deep convolutions. However, high-resolution remote sensing CD remains challenging due to the complexity of objects in the scene. Objects with the same semantic concept may show distinct spectral characteristics at different times and spatial locations. Most recent CD pipelines using pure convolutions are still struggling to relate long-range concepts in space-time. Nonlocal self-attention approaches show promising performance via modeling dense relationships among pixels, yet are computationally inefficient. Here, we propose a bitemporal image transformer (BIT) to efficiently and effectively model contexts within the spatial-temporal domain. Our intuition is that the high-level concepts of the change of interest can be represented by a few visual words, that is, semantic tokens. To achieve this, we express the bitemporal image into a few tokens and use a transformer encoder to model contexts in the compact token-based space-time. The learned context-rich tokens are then fed back to the pixel-space for refining the original features via a transformer decoder. We incorporate BIT in a deep feature differencing-based CD framework. Extensive experiments on three CD datasets demonstrate the effectiveness and efficiency of the proposed method. Notably, our BIT-based model significantly outperforms the purely convolutional baseline using only three times lower computational costs and model parameters. Based on a naive backbone (ResNet18) without sophisticated structures (e.g., feature pyramid network (FPN) and UNet), our model surpasses several state-of-the-art CD methods, including better than four recent attention-based methods in terms of efficiency and accuracy. Our code is available at https://github.com/justchenhao/BIT_CD.
Better SGD using Second-order Momentum

We develop a new algorithm for non-convex stochastic optimization that finds an $\epsilon$-critical point in the optimal $O(\epsilon^{-3})$ stochastic gradient and Hessian-vector product computations. Our algorithm uses Hessian-vector products to"correct"a bias term in the momentum of SGD with momentum. This leads to better gradient estimates in a manner analogous to variance reduction methods. In contrast to prior work, we do not require excessively large batch sizes, and are able to provide an adaptive algorithm whose convergence rate automatically improves with decreasing variance in the gradient estimates. We validate our results on a variety of large-scale deep learning architectures and benchmarks tasks.
Monitoring In Situ Biodegradation of MTBE Using Multiple Rounds of Compound‐Specific Stable Carbon Isotope Analysis

At a large industrial facility, methyl tert‐butyl ether (MTBE) was released to the subsurface and dispersed into the light, non‐aqueous phase liquids (LNAPL), in the first aquifer, with the LNAPL serving as a continuous source of MTBE in groundwater. Compound‐specific isotope analysis was conducted on both MTBE and tert‐butyl alcohol (TBA) in groundwater samples collected in 2008, 2011, and 2013 from wells located along and off the center line of the MTBE plume. The study demonstrated the onset and progress of biodegradation of MTBE between 2008 and 2013. The TBA observed in 2008 appears to be derived only in part from MTBE transformation while a significant portion of TBA might be contributed directly from LNAPL sources. In 2011 to 2013, the dominant source of TBA in the mid‐gradient plume was MTBE transformation. A contribution of an offsite LNAPL source, in particular to the down‐gradient area of the plume, is possible but could not be unequivocally confirmed. The time series provided direct evidence for MTBE biodegradation, but also a valuable insight on the sources of TBA.
Accuracy on the Line: on the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization

For machine learning systems to be reliable, we must understand their performance in unseen, out-of-distribution environments. In this paper, we empirically show that out-of-distribution performance is strongly correlated with in-distribution performance for a wide range of models and distribution shifts. Specifically, we demonstrate strong correlations between in-distribution and out-of-distribution performance on variants of CIFAR-10&ImageNet, a synthetic pose estimation task derived from YCB objects, satellite imagery classification in FMoW-WILDS, and wildlife classification in iWildCam-WILDS. The strong correlations hold across model architectures, hyperparameters, training set size, and training duration, and are more precise than what is expected from existing domain adaptation theory. To complete the picture, we also investigate cases where the correlation is weaker, for instance some synthetic distribution shifts from CIFAR-10-C and the tissue classification dataset Camelyon17-WILDS. Finally, we provide a candidate theory based on a Gaussian data model that shows how changes in the data covariance arising from distribution shift can affect the observed correlations.
Sustained cost declines in solar PV and battery storage needed to eliminate coal generation in India

Unabated coal power in India must be phased out by mid-century to achieve global climate targets under the Paris Agreement. Here we estimate the costs of hybrid power plants—lithium-ion battery storage with wind and solar PV—to replace coal generation. We design least cost mixes of these technologies to supply stylized baseload and load-following generation profiles in three Indian states—Karnataka, Gujarat, and Tamil Nadu. Our analysis shows that availability of low cost capital, solar PV capital costs of at least $250 kW−1, and battery storage capacity costs at least 50% cheaper than current levels will be required to phase out existing coal power plants. Phaseout by 2040 requires a 6% annual decline in the levelized cost of hybrid systems over the next two decades. We find that replacing coal generation with hybrid systems 99% of the hours over multiple decades is roughly 40% cheaper than 100% replacement, indicating a key role for other low cost grid flexibility mechanisms to help hasten coal phaseout. Solar PV is more suited to pairing with short duration storage than wind power. Overall, our results describe the challenging technological and policy advances needed to achieve the temperature goals of the Paris Agreement.
Deep Quantile Regression: Mitigating the Curse of Dimensionality Through Composition

This paper considers the problem of nonparametric quantile regression under the assumption that the target conditional quantile function is a composition of a sequence of low-dimensional functions. We study the nonparametric quantile regression estimator using deep neural networks to approximate the target conditional quantile function. For convenience, we shall refer to such an estimator as a deep quantile regression (DQR) estimator. We establish non-asymptotic error bounds for the excess risk and the mean integrated squared errors of the DQR estimator. Our results show that the DQR estimator has an oracle property in the sense that it achieves the nonparametric minimax optimal rate determined by the intrinsic dimension of the underlying compositional structure of the conditional quantile function, not the ambient dimension of the predictor. Therefore, DQR is able to mitigate the curse of dimensionality under the assumption that the conditional quantile function has a compositional structure. To establish these results, we analyze the approximation error of a composite function by neural networks and show that the error rate only depends on the dimensions of the component functions, instead of the ambient dimension of the function. We apply our general results to several important statistical models often used in mitigating the curse of dimensionality, including the single index, the additive, the projection pursuit, the univariate composite, and the generalized hierarchical interaction models. We explicitly describe the prefactors in the error bounds in terms of the dimensionality of the data and show that the prefactors depends on the dimensionality linearly or quadratically in these models.
Combining p-multigrid and multigrid reduced in time methods to obtain a scalable solver for Isogeometric Analysis

Isogeometric Analysis (IgA) has become a viable alternative to the Finite Element Method (FEM) and is typically combined with a time integration scheme within the method of lines for time-dependent problems. However, due to a stagnation of processors clock speeds, traditional (i.e. sequential) time integration schemes become more and more the bottleneck within these large-scale computations, which lead to the development of parallel-in-time methods like the Multigrid Reduced in Time (MGRIT) method. Recently, MGRIT has been succesfully applied by the authors in the context of IgA showing convergence independent of the mesh width, approximation order of the B-spline basis functions and time step size for a variety of benchmark problems. However, a strong dependency of the CPU times on the approximation order was visible when a standard Conjugate Gradient method was adopted for the spatial solves within MGRIT. In this paper we combine MGRIT with a state-of the-art solver (i.e. a p-multigrid method), specifically designed for IgA, thereby significantly reducing the overall computational costs of MGRIT. Furthermore, we investigate the performance of MGRIT and its scalability on modern copmuter architectures.
Exploring low lattice thermal conductivity materials using chemical bonding principles

Semiconductors with very low lattice thermal conductivities are highly desired for applications relevant to thermal energy conversion and management, such as thermoelectrics and thermal barrier coatings. Although the crystal structure and chemical bonding are known to play vital roles in shaping heat transfer behavior, material design approaches of lowering lattice thermal conductivity using chemical bonding principles are uncommon. In this work, we present an effective strategy of weakening interatomic interactions and therefore suppressing lattice thermal conductivity based on chemical bonding principles and develop a high-efficiency approach of discovering low κL materials by screening the local coordination environments of crystalline compounds. The followed first-principles calculations uncover 30 hitherto unexplored compounds with (ultra)low lattice thermal conductivities from thirteen prototype crystal structures contained in the inorganic crystal structure database. Furthermore, we demonstrate an approach of rationally designing high-performance thermoelectrics by additionally incorporating cations with stereochemically active lone-pair electrons. Our results not only provide fundamental insights into the physical origin of the low lattice thermal conductivity in a large family of copper-based compounds but also offer an efficient approach to discovery and design materials with targeted thermal transport properties.