Miracle-dz/UniFAR-FTUs huggingface dataset:
Modeling and forecasting the outcomes of NBA basketball games

This paper treats the problem of modeling and forecasting the outcomes of NBA basketball games. First, it is shown how the benchmark model in the literature can be extended to allow for heteroscedasticity and estimation and testing in this framework is treated. Second, time-variation is introduced into the model by introducing a dynamic state space model for team strengths. The in-sample results based on eight seasons of NBA data provide weak evidence for heteroscedasticity, which can lead to notable differences in estimated win probabilities. However, persistent time variation is only found when combining the data of several seasons, but not when looking at individual seasons. The models are used for forecasting a large number of regular season and playoff games and the common finding in the literature that it is difficult to outperform the betting market is confirmed. Nevertheless, a forecast combination of model based forecasts with betting odds can lead to some slight improvements.
Robust Fault Estimators for Nonlinear Systems: An Ultra-Local Model Design

This paper proposes a nonlinear estimator for the robust reconstruction of process and sensor faults for a class of uncertain nonlinear systems. The proposed fault estimation method augments the system dynamics with an ultra-local (in time) internal state-space representation (a finite chain of integrators) of the fault vector. Next, a nonlinear state observer is designed based on the known parts of the augmented dynamics. This nonlinear filter (observer) reconstructs the fault signal as well as the states of the augmented system. We provide sufficient conditions that guarantee stability of the estimation error dynamics: firstly, asymptotic stability (i.e., perfect fault estimation) in the absence of perturbations induced by fault model mismatch (mismatch between internal, ultralocal model for the fault and the actual fault characteristics), uncertainty, external disturbances, and measurement noise and, secondly, Input-to-State Stability (ISS) of the estimation error dynamics is guaranteed in the presence of these perturbations. In addition, to support performance-based estimator design, we provide Linear Matrix Inequality (LMI) conditions for L2-gain and L2 - L_inf induced norm and cast the synthesis of the estimator gains as a semi-definite program where the effect of model mismatch and external disturbances on the fault estimation error is minimized in the sense of L2-gain, for an acceptable L2 - L_inf induced norm with respect to measurement noise. The latter result facilitates a design that explicitly addresses the performance trade-off between noise sensitivity and robustness against model mismatch and external disturbances. Finally, numerical results for a benchmark system illustrate the performance of the proposed methodologies.
Revisiting Subgradient Method: Complexity and Convergence Beyond Lipschitz Continuity

The subgradient method is one of the most fundamental algorithmic schemes for nonsmooth optimization. The existing complexity and convergence results for this algorithm are mainly derived for Lipschitz continuous objective functions. In this work, we first extend the typical complexity results for the subgradient method to convex and weakly convex minimization without assuming Lipschitz continuity. Specifically, we establish $\mathcal{O}(1/\sqrt{T})$ bound in terms of the suboptimality gap ``$f(x) - f^*$'' for convex case and $\mathcal{O}(1/{T}^{1/4})$ bound in terms of the gradient of the Moreau envelope function for weakly convex case. Furthermore, we provide convergence results for non-Lipschitz convex and weakly convex objective functions using proper diminishing rules on the step sizes. In particular, when $f$ is convex, we show $\mathcal{O}(\log(k)/\sqrt{k})$ rate of convergence in terms of the suboptimality gap. With an additional quadratic growth condition, the rate is improved to $\mathcal{O}(1/k)$ in terms of the squared distance to the optimal solution set. When $f$ is weakly convex, asymptotic convergence is derived. The central idea is that the dynamics of properly chosen step sizes rule fully controls the movement of the subgradient method, which leads to boundedness of the iterates, and then a trajectory-based analysis can be conducted to establish the desired results. To further illustrate the wide applicability of our framework, we extend the complexity results to the truncated subgradient, the stochastic subgradient, the incremental subgradient, and the proximal subgradient methods for non-Lipschitz functions.
Query Rewriting for Retrieval-Augmented Large Language Models

Large Language Models (LLMs) play powerful, black-box readers in the retrieve-then-read pipeline, making remarkable progress in knowledge-intensive tasks. This work introduces a new framework, Rewrite-Retrieve-Read instead of the previous retrieve-then-read for the retrieval-augmented LLMs from the perspective of the query rewriting. Unlike prior studies focusing on adapting either the retriever or the reader, our approach pays attention to the adaptation of the search query itself, for there is inevitably a gap between the input text and the needed knowledge in retrieval. We first prompt an LLM to generate the query, then use a web search engine to retrieve contexts. Furthermore, to better align the query to the frozen modules, we propose a trainable scheme for our pipeline. A small language model is adopted as a trainable rewriter to cater to the black-box LLM reader. The rewriter is trained using the feedback of the LLM reader by reinforcement learning. Evaluation is conducted on downstream tasks, open-domain QA and multiple-choice QA. Experiments results show consistent performance improvement, indicating that our framework is proven effective and scalable, and brings a new framework for retrieval-augmented LLM.
The neural dynamics of auditory word recognition and integration

Listeners recognize and integrate words in rapid and noisy everyday speech by combining expectations about upcoming content with incremental sensory evidence. We present a computational model of word recognition which formalizes this perceptual process in Bayesian decision theory. We fit this model to explain scalp EEG signals recorded as subjects passively listened to a fictional story, revealing both the dynamics of the online auditory word recognition process and the neural correlates of the recognition and integration of words. The model reveals distinct neural processing of words depending on whether or not they can be quickly recognized. While all words trigger a neural response characteristic of probabilistic integration -- voltage modulations predicted by a word's surprisal in context -- these modulations are amplified for words which require more than roughly 150 ms of input to be recognized. We observe no difference in the latency of these neural responses according to words' recognition times. Our results are consistent with a two-part model of speech comprehension, combining an eager and rapid process of word recognition with a temporally independent process of word integration. However, we also developed alternative models of the scalp EEG signal not incorporating word recognition dynamics which showed similar performance improvements. We discuss potential future modeling steps which may help to separate these hypotheses.
Causation, Comparison, and Regression

Comparison and contrast are the basic means to unveil causation and learn which treatments work. To build good comparison groups, randomized experimentation is key, yet often infeasible. In such non-experimental settings, we illustrate and discuss diagnostics to assess how well the common linear regression approach to causal inference approximates desirable features of randomized experiments, such as covariate balance, study representativeness, interpolated estimation, and unweighted analyses. We also discuss alternative regression modeling, weighting, and matching approaches and argue they should be given strong consideration in empirical work.
mPMR: A Multilingual Pre-trained Machine Reader at Scale

We present multilingual Pre-trained Machine Reader (mPMR), a novel method for multilingual machine reading comprehension (MRC)-style pre-training. mPMR aims to guide multilingual pre-trained language models (mPLMs) to perform natural language understanding (NLU) including both sequence classification and span extraction in multiple languages. To achieve cross-lingual generalization when only source-language fine-tuning data is available, existing mPLMs solely transfer NLU capability from a source language to target languages. In contrast, mPMR allows the direct inheritance of multilingual NLU capability from the MRC-style pre-training to downstream tasks. Therefore, mPMR acquires better NLU capability for target languages. mPMR also provides a unified solver for tackling cross-lingual span extraction and sequence classification, thereby enabling the extraction of rationales to explain the sentence-pair classification process.
DiffHand: End-to-End Hand Mesh Reconstruction via Diffusion Models

Hand mesh reconstruction from the monocular image is a challenging task due to its depth ambiguity and severe occlusion, there remains a non-unique mapping between the monocular image and hand mesh. To address this, we develop DiffHand, the first diffusion-based framework that approaches hand mesh reconstruction as a denoising diffusion process. Our one-stage pipeline utilizes noise to model the uncertainty distribution of the intermediate hand mesh in a forward process. We reformulate the denoising diffusion process to gradually refine noisy hand mesh and then select mesh with the highest probability of being correct based on the image itself, rather than relying on 2D joints extracted beforehand. To better model the connectivity of hand vertices, we design a novel network module called the cross-modality decoder. Extensive experiments on the popular benchmarks demonstrate that our method outperforms the state-of-the-art hand mesh reconstruction approaches by achieving 5.8mm PA-MPJPE on the Freihand test set, 4.98mm PA-MPJPE on the DexYCB test set.
Know Your Self-supervised Learning: A Survey on Image-based Generative and Discriminative Training

Although supervised learning has been highly successful in improving the state-of-the-art in the domain of image-based computer vision in the past, the margin of improvement has diminished significantly in recent years, indicating that a plateau is in sight. Meanwhile, the use of self-supervised learning (SSL) for the purpose of natural language processing (NLP) has seen tremendous successes during the past couple of years, with this new learning paradigm yielding powerful language models. Inspired by the excellent results obtained in the field of NLP, self-supervised methods that rely on clustering, contrastive learning, distillation, and information-maximization, which all fall under the banner of discriminative SSL, have experienced a swift uptake in the area of computer vision. Shortly afterwards, generative SSL frameworks that are mostly based on masked image modeling, complemented and surpassed the results obtained with discriminative SSL. Consequently, within a span of three years, over $100$ unique general-purpose frameworks for generative and discriminative SSL, with a focus on imaging, were proposed. In this survey, we review a plethora of research efforts conducted on image-oriented SSL, providing a historic view and paying attention to best practices as well as useful software packages. While doing so, we discuss pretext tasks for image-based SSL, as well as techniques that are commonly used in image-based SSL. Lastly, to aid researchers who aim at contributing to image-focused SSL, we outline a number of promising research directions.
On the role of eigenvalue disparity and coordinate transformations in the reduction of the linear noise approximation

Eigenvalue disparity, also known as timescale separation, permits the systematic reduction of deterministic models of enzyme kinetics. Geometric singular perturbation theory, of which eigenvalue disparity is central, provides a coordinate-free framework for deriving reduced mass action models in the deterministic realm. Moreover, homologous deterministic reductions are often employed in stochastic models to reduce the computational complexity required to simulate reactions with the Gillespie algorithm. Interestingly, several detailed studies indicate that timescale separation does not always guarantee the accuracy of reduced stochastic models. In this work, we examine the roles of timescale separation and coordinate transformations in the reduction of the Linear Noise Approximation (LNA) and, unlike previous studies, we do not require the system to be comprised of distinct fast and slow variables. Instead, we adopt a coordinate-free approach. We demonstrate that eigenvalue disparity does not guarantee the accuracy of the reduced LNA, known as the slow scale LNA (ssLNA). However, the inaccuracy of the ssLNA can often be eliminated with a proper coordinate transformation. For planar systems in separated (standard) form, we prove that the error between the variances of the slow variable generated by the LNA and the ssLNA is $\mathcal{O}(\varepsilon)$. We also address a nilpotent Jacobian scenario and use the blow-up method to construct a reduced equation that is accurate near the singular limit in the deterministic regime. However, this reduction in the stochastic regime is far less accurate, which illustrates that eigenvalue disparity plays a central role in stochastic model reduction.
Correcting Airborne Temperature Data for Lags Introduced by Instruments with Two-Time-Constant Responses

Abstract Airborne resistance wire temperature sensors can introduce a time lag before ambient air temperature changes are registered. It has been found that a second-order linear response system adequately describes the behavior of the Rosemount non-deiced 102 series instrument consisting of a platinum resistance wire sensor with a protective housing. A method has been proposed in the literature to deconvolve measured temperatures based on this system to obtain the ambient temperature. Using the exact solution of the underlying differential equation formulation of that method, a more efficient and more accurate correction algorithm is developed here. As with its predecessor, this new algorithm can be applied to temperature data obtained in dry conditions and so is suitable for dry boundary layer but not in-cloud measurements.
Wikipedia and open access

Wikipedia is a well-known platform for disseminating knowledge, and scientific sources, such as journal articles, play a critical role in supporting its mission. The open access movement aims to make scientific knowledge openly available, and we might intuitively expect open access to help further Wikipedia's mission. However, the extent of this relationship remains largely unknown. To fill this gap, we analyze a large dataset of citations from Wikipedia and model the role of open access in Wikipedia's citation patterns. We find that open-access articles are extensively and increasingly more cited in Wikipedia. What is more, they show a 15% higher likelihood of being cited in Wikipedia when compared to closed-access articles, after controlling for confounding factors. This open-access citation effect is particularly strong for articles with low citation counts, including recently published ones. Our results show that open access plays a key role in the dissemination of scientific knowledge, including by providing Wikipedia editors timely access to novel results. These findings have important implications for researchers, policymakers, and practitioners in the field of information science and technology.
High order asymptotic preserving scheme for linear kinetic equations with diffusive scaling

In this work, high order asymptotic preserving schemes are constructed and analysed for kinetic equations under a diffusive scaling. The framework enables to consider different cases: the diffusion equation, the advection-diffusion equation and the presence of inflow boundary conditions. Starting from the micro-macro reformulation of the original kinetic equation, high order time integrators are introduced. This class of numerical schemes enjoys the Asymptotic Preserving (AP) property for arbitrary initial data and degenerates when $\epsilon$ goes to zero into a high order scheme which is implicit for the diffusion term, which makes it free from the usual diffusion stability condition. The space discretization is also discussed and high order methods are also proposed based on classical finite differences schemes. The Asymptotic Preserving property is analysed and numerical results are presented to illustrate the properties of the proposed schemes in different regimes.
NCC: Natural Concurrency Control for Strictly Serializable Datastores by Avoiding the Timestamp-Inversion Pitfall

Strictly serializable datastores greatly simplify the development of correct applications by providing strong consistency guarantees. However, existing techniques pay unnecessary costs for naturally consistent transactions, which arrive at servers in an order that is already strictly serializable. We find these transactions are prevalent in datacenter workloads. We exploit this natural arrival order by executing transaction requests with minimal costs while optimistically assuming they are naturally consistent, and then leverage a timestamp-based technique to efficiently verify if the execution is indeed consistent. In the process of designing such a timestamp-based technique, we identify a fundamental pitfall in relying on timestamps to provide strict serializability, and name it the timestamp-inversion pitfall. We find timestamp-inversion has affected several existing works. We present Natural Concurrency Control (NCC), a new concurrency control technique that guarantees strict serializability and ensures minimal costs -- i.e., one-round latency, lock-free, and non-blocking execution -- in the best (and common) case by leveraging natural consistency. NCC is enabled by three key components: non-blocking execution, decoupled response control, and timestamp-based consistency check. NCC avoids timestamp-inversion with a new technique: response timing control, and proposes two optimization techniques, asynchrony-aware timestamps and smart retry, to reduce false aborts. Moreover, NCC designs a specialized protocol for read-only transactions, which is the first to achieve the optimal best-case performance while ensuring strict serializability, without relying on synchronized clocks. Our evaluation shows that NCC outperforms state-of-the-art solutions by an order of magnitude on many workloads.
TaDSE: Template-aware Dialogue Sentence Embeddings

Learning high quality sentence embeddings from dialogues has drawn increasing attentions as it is essential to solve a variety of dialogue-oriented tasks with low annotation cost. However, directly annotating and gathering utterance relationships in conversations are difficult, while token-level annotations, \eg, entities, slots and templates, are much easier to obtain. General sentence embedding methods are usually sentence-level self-supervised frameworks and cannot utilize token-level extra knowledge. In this paper, we introduce Template-aware Dialogue Sentence Embedding (TaDSE), a novel augmentation method that utilizes template information to effectively learn utterance representation via self-supervised contrastive learning framework. TaDSE augments each sentence with its corresponding template and then conducts pairwise contrastive learning over both sentence and template. We further enhance the effect with a synthetically augmented dataset that enhances utterance-template relation, in which entity detection (slot-filling) is a preliminary step. We evaluate TaDSE performance on five downstream benchmark datasets. The experiment results show that TaDSE achieves significant improvements over previous SOTA methods, along with a consistent Intent Classification task performance improvement margin. We further introduce a novel analytic instrument of Semantic Compression method, for which we discover a correlation with uniformity and alignment. Our code will be released soon.
Ordinal Outcome State-Space Models for Intensive Longitudinal Data

Intensive longitudinal (IL) data are increasingly prevalent in psychological science, coinciding with technological advancements that make it simple to deploy study designs such as daily diary and ecological momentary assessments. IL data are characterized by a rapid rate of data collection (1+ collections per day), over a period of time, allowing for the capture of the dynamics that underlie psychological and behavioral processes. One powerful framework for analyzing IL data is state-space modeling, where observed variables are considered measurements for underlying states (i.e., latent variables) that change together over time. However, state-space modeling has typically relied on continuous measurements, whereas psychological data often comes in the form of ordinal measurements such as Likert scale items. In this manuscript, we develop a general estimating approach for state-space models with ordinal measurements, specifically focusing on a graded response model for Likert scale items. We evaluate the performance of our model and estimator against that of the commonly used ``linear approximation'' model, which treats ordinal measurements as though they are continuous. We find that our model resulted in unbiased estimates of the state dynamics, while the linear approximation resulted in strongly biased estimates of the state dynamics
Improvement of graviton mass constraints using GRAVITY’s detection of Schwarzschild precession in the orbit of S2 star around the Galactic Center

Here we study possible improvements of the existing constraints on the upper bound of graviton mass by the analysis of the stellar orbits around the SMBH at the GC in the framework of Yukawa gravity. A motivation for this study is a recent detection of Schwarzschild precession in the orbit of S2 star around the SMBH at the GC by the GRAVITY Collaboration. The authors indicated that the orbital precession of the S2 star is close to the General Relativity (GR) prediction, but with possible small deviation from it, and parametrized this effect by introducing an ad hoc factor in the parametrized PPN equations of motion. Here we use the value of this factor presented by GRAVITY in order to perform two-body simulations of the stellar orbits in massive gravity using equations of motion in the modified PPN formalism, as well as to constrain the range of massive interaction $\Lambda$. From the obtained values of $\Lambda$, and assuming that it corresponds to the Compton wavelength of graviton, we then calculated new estimates for the upper bound of graviton mass which are found to be independent, but consistent with the LIGO's estimate of graviton mass from the first GW signal GW150914 (later this graviton mass estimation was significantly improved with consequent observations of GW events). We also performed calculations including numerical simulations in order to constrain the bounds on graviton mass in the case of a small deviation of the stellar orbits from the corresponding GR predictions and showed that our method could further improve previous estimates for upper bounds on the graviton mass. It is also demonstrated that such analysis of the observed orbits of S-stars around the GC in the frame of the Yukawa gravity represents a tool for constraining the upper bound for the graviton mass, as well as for probing the predictions of GR or other gravity theories.