Miracle-dz/UniFAR-FTUs huggingface dataset:
Probabilistic Design of Multi-Dimensional Spatially-Coupled Codes

Because of their excellent asymptotic and finite-length performance, spatially-coupled (SC) codes are a class of low-density parity-check codes that is gaining increasing attention. Multi-dimensional (MD) SC codes are constructed by connecting copies of an SC code via relocations in order to mitigate various sources of non-uniformity and improve performance in many data storage and data transmission systems. As the number of degrees of freedom in the MD-SC code design increases, appropriately exploiting them becomes more difficult because of the complexity growth of the design process. In this paper, we propose a probabilistic framework for the MD-SC code design, which is based on the gradient-descent (GD) algorithm, to design better MD codes and address this challenge. In particular, we express the expected number of short cycles, which we seek to minimize, in the graph representation of the code in terms of entries of a probability-distribution matrix that characterizes the MD-SC code design. We then find a locally-optimal probability distribution, which serves as the starting point of a finite-length algorithmic optimizer that produces the final MD-SC code. We offer the theoretical analysis as well as the algorithms, and we present experimental results demonstrating that our MD codes, conveniently called GD-MD codes, have notably lower short cycle numbers compared with the available state-of-the-art. Moreover, our algorithms converge on solutions in few iterations, which confirms the complexity reduction as a result of limiting the search space via the locally-optimal GD-MD distributions.
An Information-Theoretic Analysis of In-Context Learning

Previous theoretical results pertaining to meta-learning on sequences build on contrived assumptions and are somewhat convoluted. We introduce new information-theoretic tools that lead to an elegant and very general decomposition of error into three components: irreducible error, meta-learning error, and intra-task error. These tools unify analyses across many meta-learning challenges. To illustrate, we apply them to establish new results about in-context learning with transformers. Our theoretical results characterizes how error decays in both the number of training sequences and sequence lengths. Our results are very general; for example, they avoid contrived mixing time assumptions made by all prior results that establish decay of error with sequence length.
Single-Winner Voting with Alliances: Avoiding the Spoiler Effect

We study the setting of single-winner elections with ordinal preferences where candidates might be members of \emph{alliances} (which may correspond to e.g., political parties, factions, or coalitions). However, we do not assume that candidates from the same alliance are necessarily adjacent in voters' rankings. In such case, every classical voting rule is vulnerable to the spoiler effect, i.e., the presence of a candidate may harm his or her alliance. We therefore introduce a new idea of \emph{alliance-aware} voting rules which extend the classical ones. We show that our approach is superior both to using classical cloneproof voting rules and to running primaries within alliances before the election. We introduce several alliance-aware voting rules and show that they satisfy the most desirable standard properties of their classical counterparts as well as newly introduced axioms for the model with alliances which, e.g., exclude the possibility of the spoiler effect. Our rules have natural definitions and are simple enough to explain to be used in practice.
Do We Need Language-Specific Fact-Checking Models? The Case of Chinese

This paper investigates the potential benefits of language-specific fact-checking models, focusing on the case of Chinese. We first demonstrate the limitations of translation-based methods and multilingual large language models (e.g., GPT-4), highlighting the need for language-specific systems. We further propose a Chinese fact-checking system that can better retrieve evidence from a document by incorporating context information. To better analyze token-level biases in different systems, we construct an adversarial dataset based on the CHEF dataset, where each instance has large word overlap with the original one but holds the opposite veracity label. Experimental results on the CHEF dataset and our adversarial dataset show that our proposed method outperforms translation-based methods and multilingual LLMs and is more robust toward biases, while there is still large room for improvement, emphasizing the importance of language-specific fact-checking systems.
The Exceptionally Warm Winter of 2015/16 in Alaska

AbstractAlaska experienced record-setting warmth during the 2015/16 cold season (October–April). Statewide average temperatures exceeded the period-of-record mean by more than 4°C over the 7-month cold season and by more than 6°C over the 4-month late-winter period, January–April. The record warmth raises two questions: 1) Why was Alaska so warm during the 2015/16 cold season? 2) At what point in the future might this warmth become typical if greenhouse warming continues? On the basis of circulation analogs computed from sea level pressure and 850-hPa geopotential height fields, the atmospheric circulation explains less than half of the anomalous warmth. The warming signal forced by greenhouse gases in climate models accounts for about 1°C of the anomalous warmth. A factor that is consistent with the seasonal and spatial patterns of the warmth is the anomalous surface state. The surface anomalies include 1) above-normal ocean surface temperatures and below-normal sea ice coverage in the surrounding seas f.
Lower semicontinuity, Stoilow factorization and principal maps

We consider a strengthening of the usual quasiconvexity condition of Morrey in two dimensions, which allows us to prove lower semicontinuity for functionals which are unbounded as the determinant vanishes. This notion, that we call principal quasiconvexity, arose from the planar theory of quasiconformal mappings and mappings of finite distortion. We compare it with other quasiconvexity conditions that have appeared in the literature and provide a number of concrete examples of principally quasiconvex functionals that are not polyconvex. The Stoilow factorization, that in the context of maps of integrable distortion was developed by Iwaniec and \v{S}ver\'ak, plays a prominent role in our approach.
Computational Synthetic Cohomology Theory in Homotopy Type Theory

This paper discusses the development of synthetic cohomology in Homotopy Type Theory (HoTT), as well as its computer formalisation. The objectives of this paper are (1) to generalise previous work on integral cohomology in HoTT by the current authors and Brunerie (2022) to cohomology with arbitrary coefficients and (2) to provide the mathematical details of, as well as extend, results underpinning the computer formalisation of cohomology rings by the current authors and Lamiaux (2023). With respect to objective (1), we provide new direct definitions of the cohomology group operations and of the cup product, which, just as in (Brunerie et al., 2022), enable significant simplifications of many earlier proofs in synthetic cohomology theory. In particular, the new definition of the cup product allows us to give the first complete formalisation of the axioms needed to turn the cohomology groups into a graded commutative ring. We also establish that this cohomology theory satisfies the HoTT formulation of the Eilenberg-Steenrod axioms for cohomology and study the classical Mayer-Vietoris and Gysin sequences. With respect to objective (2), we characterise the cohomology groups and rings of various spaces, including the spheres, torus, Klein bottle, real/complex projective planes, and infinite real projective space. All results have been formalised in Cubical Agda and we obtain multiple new numbers, similar to the famous `Brunerie number', which can be used as benchmarks for computational implementations of HoTT. Some of these numbers are infeasible to compute in Cubical Agda and hence provide new computational challenges and open problems which are much easier to define than the original Brunerie number.
Continuous-Multiple Image Outpainting in One-Step via Positional Query and A Diffusion-based Approach

Image outpainting aims to generate the content of an input sub-image beyond its original boundaries. It is an important task in content generation yet remains an open problem for generative models. This paper pushes the technical frontier of image outpainting in two directions that have not been resolved in literature: 1) outpainting with arbitrary and continuous multiples (without restriction), and 2) outpainting in a single step (even for large expansion multiples). Moreover, we develop a method that does not depend on a pre-trained backbone network, which is in contrast commonly required by the previous SOTA outpainting methods. The arbitrary multiple outpainting is achieved by utilizing randomly cropped views from the same image during training to capture arbitrary relative positional information. Specifically, by feeding one view and positional embeddings as queries, we can reconstruct another view. At inference, we generate images with arbitrary expansion multiples by inputting an anchor image and its corresponding positional embeddings. The one-step outpainting ability here is particularly noteworthy in contrast to previous methods that need to be performed for $N$ times to obtain a final multiple which is $N$ times of its basic and fixed multiple. We evaluate the proposed approach (called PQDiff as we adopt a diffusion-based generator as our embodiment, under our proposed \textbf{P}ositional \textbf{Q}uery scheme) on public benchmarks, demonstrating its superior performance over state-of-the-art approaches. Specifically, PQDiff achieves state-of-the-art FID scores on the Scenery (\textbf{21.512}), Building Facades (\textbf{25.310}), and WikiArts (\textbf{36.212}) datasets. Furthermore, under the 2.25x, 5x and 11.7x outpainting settings, PQDiff only takes \textbf{40.6\%}, \textbf{20.3\%} and \textbf{10.2\%} of the time of the benchmark state-of-the-art (SOTA) method.
Constraining $f(T,B,T_G,B_G)$ gravity by dynamical system analysis

The evolutionary behavior of the Universe has been analysed through the dynamical system analysis in $f(T,B,T_G,B_G)$ gravity, where $T$, $B$, $T_G$, and $B_G$ respectively represent torsion, boundary term, teleparallel Gauss-Bonnet term and Gauss-Bonnet boundary term. We use the transformation, $f(T,B,T_G,B_G)=-T+\mathcal{F}(T, B, T_G, B_G)$ in order to obtain the deviation from the Teleparallel Equivalent of General Relativity (TEGR). Two cosmological models pertaining to the functional form of $\mathcal{F}(T, B, T_G, B_G)$ have been studied. The well motivated forms are: (i) $\mathcal{F}(T, B, T_G, B_G) = f_{0} T^{m} B^{n}T_{G}^{k}$ and (ii) $\mathcal{F}(T, B, T_G, B_G)=b_{0} B + g_{0} T_{G}^{k} $. The evolutionary phases of the Universe have been identified through the detailed analysis of the critical points. Further, with the eigenvalues and phase space diagrams, the stability and attractor nature of the accelerating solution have been explored. The evolution plots have been analyzed for the corresponding cosmology and compatibility with the present observed value of standard density parameters have been shown.
Semantics of Multiword Expressions in Transformer-Based Models: A Survey

Multiword expressions (MWEs) are composed of multiple words and exhibit variable degrees of compositionality. As such, their meanings are notoriously difficult to model, and it is unclear to what extent this issue affects transformer architectures. Addressing this gap, we provide the first in-depth survey of MWE processing with transformer models. We overall find that they capture MWE semantics inconsistently, as shown by reliance on surface patterns and memorized information. MWE meaning is also strongly localized, predominantly in early layers of the architecture. Representations benefit from specific linguistic properties, such as lower semantic idiosyncrasy and ambiguity of target expressions. Our findings overall question the ability of transformer models to robustly capture fine-grained semantics. Furthermore, we highlight the need for more directly comparable evaluation setups.
A Causal Model for Quantifying Multipartite Classical and Quantum Correlations

We give an operational definition of information-theoretic resources within a given multipartite classical or quantum correlation. We present our causal model that serves as the source coding side of this correlation and introduce a novel concept of resource rate. We argue that, beyond classical secrecy, additional resources exist that are useful for the security of distributed computing problems, which can be captured by the resource rate. Furthermore, we establish a relationship between resource rate and an extension of Shannon's logarithmic information measure, namely, total correlation. Subsequently, we present a novel quantum secrecy monotone and investigate a quantum hybrid key distribution system as an extension of our causal model. Finally, we discuss some connections to optimal transport (OT) problem.
Designing and Testing a Mobile Application for Collecting WhatsApp Chat Data While Preserving Privacy

It is common practice for researchers to join public WhatsApp chats and scrape their contents for analysis. However, research shows collecting data this way contradicts user expectations and preferences, even if the data is effectively public. To overcome these issues, we outline design considerations for collecting WhatsApp chat data with improved user privacy by heightening user control and oversight of data collection and taking care to minimize the data researchers collect and process off a user's device. We refer to these design principles as User-Centered Data Sharing (UCDS). To evaluate our UCDS principles, we implemented a mobile application representing one possible instance of these improved data collection techniques and evaluated the viability of using the app to collect WhatsApp chat data. Second, we surveyed WhatsApp users to gather user perceptions on common existing WhatsApp data collection methods as well as UCDS methods. Our results show that we were able to glean similar informative insights into WhatsApp chats using UCDS principles in our prototype app to common, less privacy-preserving methods. Our survey showed that methods following the UCDS principles are preferred by users because they offered users more control over the data collection process. Future user studies could further expand upon UCDS principles to overcome complications of researcher-to-group communication in research on WhatsApp chats and evaluate these principles in other data sharing contexts.
Oracle-Efficient Hybrid Online Learning with Unknown Distribution

We study the problem of oracle-efficient hybrid online learning when the features are generated by an unknown i.i. d. process and the labels are generated adversarially. Assuming access to an (offline) ERM oracle, we show that there exists a computationally efficient online predictor that achieves a regret upper bounded by $\tilde{O}(T^{\frac{3}{4}})$ for a finite-VC class, and upper bounded by $\tilde{O}(T^{\frac{p+1}{p+2}})$ for a class with $\alpha$ fat-shattering dimension $\alpha^{-p}$. This provides the first known oracle-efficient sublinear regret bounds for hybrid online learning with an unknown feature generation process. In particular, it confirms a conjecture of Lazaric and Munos (JCSS 2012). We then extend our result to the scenario of shifting distributions with $K$ changes, yielding a regret of order $\tilde{O}(T^{\frac{4}{5}}K^{\frac{1}{5}})$. Finally, we establish a regret of $\tilde{O}((K^{\frac{2}{3}}(\log|\mathcal{H}|)^{\frac{1}{3}}+K)\cdot T^{\frac{4}{5}})$ for the contextual $K$-armed bandits with a finite policy set $\mathcal{H}$, i.i. d. generated contexts from an unknown distribution, and adversarially generated costs.
Boundary touching probability and nested-path exponent for non-simple CLE

The conformal loop ensemble (CLE) has two phases: for $\kappa \in (8/3, 4]$, the loops are simple and do not touch each other or the boundary; for $\kappa \in (4,8)$, the loops are non-simple and may touch each other and the boundary. We derive the probability that the loop surrounding a given point touches the domain boundary. We also obtain the law of the conformal radius of this loop seen from the given point conditioned on the loop touching the boundary or not, refining a result of Schramm-Sheffield-Wilson (2009). As an application, we exactly evaluate the CLE counterpart of the nested-path exponent for the Fortuin-Kasteleyn (FK) random cluster model recently introduced by Song-Tan-Zhang-Jacobsen-Nienhuis-Deng (2022). This exponent describes the asymptotic behavior of the number of nested open paths in the open cluster containing the origin when the cluster is large. For Bernoulli percolation, which corresponds to $\kappa=6$, the exponent was derived recently in Song-Jacobsen-Nienhuis-Sportiello-Deng (2023) by a color switching argument. For $\kappa\neq 6$, and in particular for the FK-Ising case, our formula appears to be new. Our derivation begins with Sheffield's construction of CLE from which the quantities of interest can be expressed by radial SLE. We solve the radial SLE problem using the coupling between SLE and Liouville quantum gravity, along with the exact solvability of Liouville conformal field theory.
On the half life of $^{71}$Ge and the Gallium Anomaly

Recent discussions about the origin of the Gallium Anomaly have motivated a remeasurement of the half life of $^{71}$Ge. We have conducted three separate measurements using dedicated planar Ge detectors: one with $^{55}$Fe as a standard, one with $^{57}$Co as a standard, and one stand alone 71Ge measurement. Our results yield a half life of 11.468 +- 0.008 days, which is consistent with but significantly more precise than the currently accepted value. With this experiment, the potential explanation of the Gallium Anomaly being due to an unexpectedly long $^{71}$Ge half life has been ruled out, leaving the origin of the anomaly as an open question.
Stochastic Amortization: A Unified Approach to Accelerate Feature and Data Attribution

Many tasks in explainable machine learning, such as data valuation and feature attribution, perform expensive computation for each data point and can be intractable for large datasets. These methods require efficient approximations, and learning a network that directly predicts the desired output, which is commonly known as amortization, is a promising solution. However, training such models with exact labels is often intractable; we therefore explore training with noisy labels and find that this is inexpensive and surprisingly effective. Through theoretical analysis of the label noise and experiments with various models and datasets, we show that this approach significantly accelerates several feature attribution and data valuation methods, often yielding an order of magnitude speedup over existing approaches.
FedFair^3: Unlocking Threefold Fairness in Federated Learning

Federated Learning (FL) is an emerging paradigm in machine learning without exposing clients' raw data. In practical scenarios with numerous clients, encouraging fair and efficient client participation in federated learning is of utmost importance, which is also challenging given the heterogeneity in data distribution and device properties. Existing works have proposed different client-selection methods that consider fairness; however, they fail to select clients with high utilities while simultaneously achieving fair accuracy levels. In this paper, we propose a fair client-selection approach that unlocks threefold fairness in federated learning. In addition to having a fair client-selection strategy, we enforce an equitable number of rounds for client participation and ensure a fair accuracy distribution over the clients. The experimental results demonstrate that FedFair^3, in comparison to the state-of-the-art baselines, achieves 18.15% less accuracy variance on the IID data and 54.78% on the non-IID data, without decreasing the global accuracy. Furthermore, it shows 24.36% less wall-clock training time on average.
Improving Medical Reasoning through Retrieval and Self-Reflection with Retrieval-Augmented Large Language Models

Recent proprietary large language models (LLMs), such as GPT-4, have achieved a milestone in tackling diverse challenges in the biomedical domain, ranging from multiple-choice questions to long-form generations. To address challenges that still cannot be handled with the encoded knowledge of LLMs, various retrieval-augmented generation (RAG) methods have been developed by searching documents from the knowledge corpus and appending them unconditionally or selectively to the input of LLMs for generation. However, when applying existing methods to different domain-specific problems, poor generalization becomes apparent, leading to fetching incorrect documents or making inaccurate judgments. In this paper, we introduce Self-BioRAG, a framework reliable for biomedical text that specializes in generating explanations, retrieving domain-specific documents, and self-reflecting generated responses. We utilize 84k filtered biomedical instruction sets to train Self-BioRAG that can assess its generated explanations with customized reflective tokens. Our work proves that domain-specific components, such as a retriever, domain-related document corpus, and instruction sets are necessary for adhering to domain-related instructions. Using three major medical question-answering benchmark datasets, experimental results of Self-BioRAG demonstrate significant performance gains by achieving a 7.2% absolute improvement on average over the state-of-the-art open-foundation model with a parameter size of 7B or less. Overall, we analyze that Self-BioRAG finds the clues in the question, retrieves relevant documents if needed, and understands how to answer with information from retrieved documents and encoded knowledge as a medical expert does. We release our data and code for training our framework components and model weights (7B and 13B) to enhance capabilities in biomedical and clinical domains.
Diffusion-based graph generative methods

Being the most cutting-edge generative methods, diffusion methods have shown great advances in wide generation tasks. Among them, graph generation attracts significant research attention for its broad application in real life. In our survey, we systematically and comprehensively review on diffusion-based graph generative methods. We first make a review on three mainstream paradigms of diffusion methods, which are denoising diffusion probabilistic models, score-based genrative models, and stochastic differential equations. Then we further categorize and introduce the latest applications of diffusion models on graphs. In the end, we point out some limitations of current studies and future directions of future explorations. The summary of existing methods metioned in this survey is in https://github.com/zhejiangzhuque/Diffusion-based-Graph-Generative-Methods.
Improved Construction of Robust Gray Code

A robust Gray code, formally introduced by (Lolck and Pagh, SODA 2024), is a Gray code that additionally has the property that, given a noisy version of the encoding of an integer $j$, it is possible to reconstruct $\hat{j}$ so that $|j - \hat{j}|$ is small with high probability. That work presented a transformation that transforms a binary code $C$ of rate $R$ to a robust Gray code with rate $\Omega(R)$, where the constant in the $\Omega(\cdot)$ can be at most $1/4$. We improve upon their construction by presenting a transformation from a (linear) binary code $C$ to a robust Gray code with similar robustness guarantees, but with rate that can approach $R/2$.