Miracle-dz/UniFAR-FTUs huggingface dataset:
SILO Language Models: Isolating Legal Risk In a Nonparametric Datastore

The legality of training language models (LMs) on copyrighted or otherwise restricted data is under intense debate. However, as we show, model performance significantly degrades if trained only on low-risk text (e.g., out-of-copyright books or government documents), due to its limited size and domain coverage. We present SILO, a new language model that manages this risk-performance tradeoff during inference. SILO is built by (1) training a parametric LM on Open License Corpus (OLC), a new corpus we curate with 228B tokens of public domain and permissively licensed text and (2) augmenting it with a more general and easily modifiable nonparametric datastore (e.g., containing copyrighted books or news) that is only queried during inference. The datastore allows use of high-risk data without training on it, supports sentence-level data attribution, and enables data producers to opt out from the model by removing content from the store. These capabilities can foster compliance with data-use regulations such as the fair use doctrine in the United States and the GDPR in the European Union. Our experiments show that the parametric LM struggles on domains not covered by OLC. However, access to the datastore greatly improves out of domain performance, closing 90% of the performance gap with an LM trained on the Pile, a more diverse corpus with mostly high-risk text. We also analyze which nonparametric approach works best, where the remaining errors lie, and how performance scales with datastore size. Our results suggest that it is possible to build high quality language models while mitigating their legal risk.
British Columbia's Commercial Salmon Industry

The salmon industry in British Columbia includes the commercial and recreational sectors as well as the salmon farming industry. Catches in the commercial fishery have decline from historic high levels of 107,500 t in 1985 to historic low levels of 17,000 t in 1999. This decline has resulted in the commercial fishing fleet being reduced by 50% over the past five years. The recreational fishery (freshwater and saltwater) has remained fairly stable since the mid-1980s with a slight (~10-15%) decline (primarily in the saltwater sector) in economic performance since 1993. The salmon farming industry has undergone explosive growth over than past 15 years and current production is now 62,000 t or roughly three times the harvest in the commercial sector. The salmon farming industry has been constrained for the last seven years by a moratorium on new sites but is expected to grow considerably in the near future. The performance of the processing sector has for the most part mirrored the commercial fishing industry. Substantial reductions in employment and a restructuring of the processing sector have occurred in recent years. New policy initiatives by the Federal government have resulted in an increased emphasis on conservation and will ensure that commercial fisheries are more selective in their harvest. Canada’s Ocean Act and Species at Risk Act are intended to ensure resource management decisions reflect ecosystem health rather than single species management approaches and to provide protection for all species at risk not simply those that are of commercial value.
Revisiting Prompt Engineering via Declarative Crowdsourcing

Large language models (LLMs) are incredibly powerful at comprehending and generating data in the form of text, but are brittle and error-prone. There has been an advent of toolkits and recipes centered around so-called prompt engineering-the process of asking an LLM to do something via a series of prompts. However, for LLM-powered data processing workflows, in particular, optimizing for quality, while keeping cost bounded, is a tedious, manual process. We put forth a vision for declarative prompt engineering. We view LLMs like crowd workers and leverage ideas from the declarative crowdsourcing literature-including leveraging multiple prompting strategies, ensuring internal consistency, and exploring hybrid-LLM-non-LLM approaches-to make prompt engineering a more principled process. Preliminary case studies on sorting, entity resolution, and imputation demonstrate the promise of our approach
An Elekes-R\'onyai theorem for sets with few products

Given $d,n \in \mathbb{N}$, we write a polynomial $F \in \mathbb{C}[x_1,\dots,x_n]$ to be degenerate if there exist $P\in \mathbb{C}[y_1, \dots, y_{n-1}]$ and $m_j = x_1^{v_{j,1}}\dots x_n^{v_{j,n}}$ with $v_{j,1}, \dots, v_{j,n} \in \mathbb{Q}$, for every $1 \leq j \leq n-1$, such that $F = P(m_1, \dots, m_{n-1})$. Our main result shows that whenever $F$ is non-degenerate, then for every finite set $A\subseteq \mathbb{C}$ such that $|A\cdot A| \leq K|A|$, one has \[ |F(A, \dots, A)| \gg_{d,n} |A|^n 2^{-O_{d,n}((\log 2K)^{3 + o(1)})}. \] This is sharp up to a factor of $O_{d,n,K}(1)$ since we have the upper bound $|F(A,\dots,A)| \leq |A|^n$ and the fact that for every degenerate $F$ and finite set $A \subseteq \mathbb{C}$ with $|A\cdot A| \leq K|A|$, one has \[ |F(A,\dots,A)| \ll K^{O_F(1)}|A|^{n-1}. \] Our techniques rely on a variety of combinatorial and linear algebraic arguments combined with Freiman type inverse theorems and Schmidt's subspace theorem.
On the stability of Caffarelli-Kohn-Nirenberg inequality in $\R^2$

Dolbeault, Esteban and Loss \cite{DEL16} obtained an optimal rigidity result, that is, when $a<0$ and $b_{\mathrm{FS}}(a)\leq b<a+1$ the extremal function for best constant $\mathcal{S}_{a,b}>0$ of the following Caffarelli-Kohn-Nirenberg inequality is symmetry, \[ \mathcal{S}_{a,b}\left(\int_{\R^2}|x|^{-qb}|u|^q \mathrm{d}x\right)^{\frac{2}{q}} \leq \int_{\R^2}|x|^{-2a}|\nabla u|^2 \mathrm{d}x, \quad \mbox{for all}\quad u\in C^\infty_0(\R^2), \] where $b_{\mathrm{FS}}(a):=a-\frac{a}{\sqrt{a^2+1}}$, $q=\frac{2}{b-a}$. An important task is investigating the stability of critical points set $\mathcal{M}$ for this inequality. Firstly, we classify solutions of the linearized problem related to the extremals which fills the work of Felli and Schneider \cite{FS03}. When $b_{\mathrm{FS}}(a)<b<a+1$, we investigate the stability of previous inequality by using spectral estimate combined with a compactness argument that \begin{align*} \int_{\mathbb{R}^2}|x|^{-2a}|\nabla u|^2 \mathrm{d}x -\mathcal{S}_{a,b}\left(\int_{\mathbb{R}^2}|x|^{-qb}|u|^q \mathrm{d}x\right)^{\frac{2}{q}} \geq \mathcal{B} \mathrm{dist}(u,\mathcal{M})^2,\quad \mbox{for all}\quad u\in C^\infty_0(\R^2), \end{align*} for some $\mathcal{B}>0$, however it is false when $b=b_{\mathrm{FS}}(a)$, which extends the work of Wei and Wu \cite{WW22} to $\R^2$. Furthermore, we obtain the existence of minimizers for $\mathcal{B}$ which extends the recent work of K\"{o}nig \cite{Ko22-2}.
Nest-DGIL: Nesterov-Optimized Deep Geometric Incremental Learning for CS Image Reconstruction

Proximal gradient-based optimization is one of the most common strategies to solve inverse problem of images, and it is easy to implement. However, these techniques often generate heavy artifacts in image reconstruction. One of the most popular refinement methods is to fine-tune the regularization parameter to alleviate such artifacts, but it may not always be sufficient or applicable due to increased computational costs. In this work, we propose a deep geometric incremental learning framework based on the second Nesterov proximal gradient optimization. The proposed end-to-end network not only has the powerful learning ability for high-/low-frequency image features, but also can theoretically guarantee that geometric texture details will be reconstructed from preliminary linear reconstruction. Furthermore, it can avoid the risk of intermediate reconstruction results falling outside the geometric decomposition domains and achieve fast convergence. Our reconstruction framework is decomposed into four modules including general linear reconstruction, cascade geometric incremental restoration, Nesterov acceleration, and post-processing. In the image restoration step, a cascade geometric incremental learning module is designed to compensate for missing texture information from different geometric spectral decomposition domains. Inspired by the overlap-tile strategy, we also develop a post-processing module to remove the block effect in patch-wise-based natural image reconstruction. All parameters in the proposed model are learnable, an adaptive initialization technique of physical parameters is also employed to make model flexibility and ensure converging smoothly. We compare the reconstruction performance of the proposed method with existing state-of-the-art methods to demonstrate its superiority.
Gentopia: A Collaborative Platform for Tool-Augmented LLMs

Augmented Language Models (ALMs) empower large language models with the ability to use tools, transforming them into intelligent agents for real-world interactions. However, most existing frameworks for ALMs, to varying degrees, are deficient in the following critical features: flexible customization, collaborative democratization, and holistic evaluation. We present gentopia, an ALM framework enabling flexible customization of agents through simple configurations, seamlessly integrating various language models, task formats, prompting modules, and plugins into a unified paradigm. Furthermore, we establish gentpool, a public platform enabling the registration and sharing of user-customized agents. Agents registered in gentpool are composable such that they can be assembled together for agent collaboration, advancing the democratization of artificial intelligence. To ensure high-quality agents, gentbench, an integral component of gentpool, is designed to thoroughly evaluate user-customized agents across diverse aspects such as safety, robustness, efficiency, etc. We release gentopia on Github and will continuously move forward.
Federated Zeroth-Order Optimization using Trajectory-Informed Surrogate Gradients

Federated optimization, an emerging paradigm which finds wide real-world applications such as federated learning, enables multiple clients (e.g., edge devices) to collaboratively optimize a global function. The clients do not share their local datasets and typically only share their local gradients. However, the gradient information is not available in many applications of federated optimization, which hence gives rise to the paradigm of federated zeroth-order optimization (ZOO). Existing federated ZOO algorithms suffer from the limitations of query and communication inefficiency, which can be attributed to (a) their reliance on a substantial number of function queries for gradient estimation and (b) the significant disparity between their realized local updates and the intended global updates. To this end, we (a) introduce trajectory-informed gradient surrogates which is able to use the history of function queries during optimization for accurate and query-efficient gradient estimation, and (b) develop the technique of adaptive gradient correction using these gradient surrogates to mitigate the aforementioned disparity. Based on these, we propose the federated zeroth-order optimization using trajectory-informed surrogate gradients (FZooS) algorithm for query- and communication-efficient federated ZOO. Our FZooS achieves theoretical improvements over the existing approaches, which is supported by our real-world experiments such as federated black-box adversarial attack and federated non-differentiable metric optimization.
Tipping points in overturning circulation mediated by ocean mixing and the configuration and magnitude of the hydrological cycle: A simple model

The current configuration of the ocean overturning involves upwelling predominantly in the Southern Ocean and sinking predominantly in the Atlantic basin. The reasons for this remain unclear, as both models and paleoclimatic observations suggest that sinking can sometimes occur in the Pacific. We present a six-box model of the overturning in which temperature, salinity and low-latitude pycnocline depths are allowed to vary prognostically in both the Atlantic and Pacific. The overturning is driven by temperature, winds, and mixing and modulated by the hydrological cycle. In each basin there are three possible flow regimes, depending on whether low-latitude water flowing into northern surface boxes is transformed into dense deep water, somewhat lighter intermediate water, or light water that is returned at the surface. The resulting model combines insights from a number of previous studies and allows for nine possible global flow regimes. For the modern ocean, we find that although the interbasin atmospheric freshwater flux suppresses Pacific sinking, the equator-to-pole flux enhances it. When atmospheric temperatures are held fixed, seven possible flow regimes can be accessed by changing the amplitude and configuration of the modern hydrological cycle . North Pacific overturning can strengthen with either increases or decreases in the hydrological cycle, as well as under reversal of the interbasin freshwater flux. Tipping-point behavior of both transient and equilibrium states is modulated by parameters such as the poorly constrained lateral diffusive mixing. If hydrological cycle amplitude is varied consistently with global temperature, northern polar amplification is necessary for the Atlantic overturning to collapse
On Monotonic Aggregation for Open-domain QA

Question answering (QA) is a critical task for speech-based retrieval from knowledge sources, by sifting only the answers without requiring to read supporting documents. Specifically, open-domain QA aims to answer user questions on unrestricted knowledge sources. Ideally, adding a source should not decrease the accuracy, but we find this property (denoted as"monotonicity") does not hold for current state-of-the-art methods. We identify the cause, and based on that we propose Judge-Specialist framework. Our framework consists of (1) specialist retrievers/readers to cover individual sources, and (2) judge, a dedicated language model to select the final answer. Our experiments show that our framework not only ensures monotonicity, but also outperforms state-of-the-art multi-source QA methods on Natural Questions. Additionally, we show that our models robustly preserve the monotonicity against noise from speech recognition. We publicly release our code and setting.
Enabling Data Confidentiality with Public Blockchains

Blockchain technology is apt to facilitate the automation of multi-party cooperations among various players in a decentralized setting, especially in cases where trust among participants is limited. Transactions are stored in a ledger, a replica of which is retained by every node of the blockchain network. The operations saved thereby are thus publicly accessible. While this aspect enhances transparency, reliability, and persistence, it hinders the utilization of public blockchains for process automation as it violates typical confidentiality requirements in corporate settings. To overcome this issue, we propose our approach named Multi-Authority Approach to Transaction Systems for Interoperating Applications (MARTSIA). Based on Multi-Authority Attribute-Based Encryption (MA-ABE), MARTSIA enables read-access control over shared data at the level of message parts. User-defined policies determine whether an actor can interpret the publicly stored information or not, depending on the actor's attributes declared by a consortium of certifiers. Still, all nodes in the blockchain network can attest to the publication of the (encrypted) data. We provide a formal analysis of the security guarantees of MARTSIA, and illustrate the proof-of-concept implementation over multiple blockchain platforms. To demonstrate its interoperability, we showcase its usage in ensemble with a state-of-the-art blockchain-based engine for multi-party process execution, and three real-world decentralized applications in the context of NFT markets, supply chain, and retail.
Motivations for religious tourism, pilgrimage, festivals and events

This chapter reassesses theories of motivation, both historical and contemporary, related to religious tourism, pilgrimage, festivals and events, recognizing the act of travel as well as the arrival as part of the complex matrix of travel motivators. The chapter introduces discourses on content theory and process theory, highlighting the work of Maslow and Herzberg to underpin the analysis of motivations for religious tourism and pilgrimage
Cumulative Reasoning with Large Language Models

While language models are powerful and versatile, they often fail to address highly complex problems. This is because solving complex problems requires deliberate thinking, which has been only minimally guided during training. In this paper, we propose a new method called Cumulative Reasoning (CR), which employs language models in a cumulative and iterative manner to emulate human thought processes. By decomposing tasks into smaller components, CR streamlines the problem-solving process, rendering it both more manageable and effective. For logical inference tasks, CR consistently outperforms existing methods with an improvement up to 9.3%, and achieves an accuracy of 98.04% on the curated FOLIO wiki dataset. In the context of the Game of 24, CR achieves an accuracy of 98%, which signifies a substantial enhancement of 24% over the previous state-of-the-art method. Finally, on the MATH dataset, we establish new state-of-the-art results with 58.0% overall accuracy, surpassing the previous best approach by a margin of 4.2%, and achieving 43% relative improvement on the hardest level 5 problems (22.4% to 32.1%). Additionally, we expand the concept of Cumulative Reasoning to incorporate a Python code environment, deliberately omitting external aids such as retrieval and web browsing and focusing solely on the LLM's intrinsic reasoning capabilities within a Python code environment. Our experiments in this setting yielded impressive results, with an overall accuracy of 72.2% on the MATH dataset, significantly outperforming the PAL method with 38.8% relative improvement. Code is available at https://github.com/iiis-ai/cumulative-reasoning.
Drainage-Basin Characteristics of Nordaustlandet Ice Caps, Svalbard

Abstract Recent mapping of ice-surface and bedrock topography from airborne radio-echo sounding has shown that the ice caps of Nordaustlandet, Svalbard, are divided into a series of well-defined drainage basins. Three lines of evidence indicate that several distinctive modes of ice-flow regime characterize these basins: (1) comparison between observed and theoretical ice-surface profiles; (2) analysis of driving stresses; and (3) observations of ice-surface features on satellite imagery and air photographs. The drainage basins are inferred to behave in the following ways. First, basins with low driving stresses and surface profiles, some of them clearly stagnant, are associated with the quiescent phase between glacier surges. Secondly, the ice streams draining southern Vestfonna have low surface profiles, relatively low driving stresses, and marked shear zones at their margins. They are interpreted to be flowing continuously at a relatively faster rate than the ridges between them. Basal melting, perhaps combined with substrate deformation, is probably responsible for the regime of these glaciers. Thirdly, the remaining basins studied on Nordaustlandet have relatively high marginal driving stresses and high surface profiles. They are interpreted to be frozen to their beds, at least near their margins. Some of these basins may also surge, particularly those where a part of the basin is below sea-level, and therefore is probably underlain by considerable thicknesses of deformable sediments.
Learning Unbiased Image Segmentation: A Case Study with Plain Knee Radiographs

Automatic segmentation of knee bony anatomy is essential in orthopedics, and it has been around for several years in both pre-operative and post-operative settings. While deep learning algorithms have demonstrated exceptional performance in medical image analysis, the assessment of fairness and potential biases within these models remains limited. This study aims to revisit deep learning-powered knee-bony anatomy segmentation using plain radiographs to uncover visible gender and racial biases. The current contribution offers the potential to advance our understanding of biases, and it provides practical insights for researchers and practitioners in medical imaging. The proposed mitigation strategies mitigate gender and racial biases, ensuring fair and unbiased segmentation results. Furthermore, this work promotes equal access to accurate diagnoses and treatment outcomes for diverse patient populations, fostering equitable and inclusive healthcare provision.
Toward Zero-Shot Instruction Following

This work proposes a challenging yet more realistic setting for zero-shot cross-task generalization: zero-shot instruction following, presuming the existence of a paragraph-style task definition while no demonstrations exist. To better learn the task supervision from the definition, we propose two strategies: first, to automatically find out the critical sentences in the definition; second, a ranking objective to force the model to generate the gold outputs with higher probabilities when those critical parts are highlighted in the definition. The joint efforts of the two strategies yield state-of-the-art performance on the Super-NaturalInstructions. Our code is available on GitHub.
GraPhSyM: Graph Physical Synthesis Model

In this work, we introduce GraPhSyM, a Graph Attention Network (GATv2) model for fast and accurate estimation of post-physical synthesis circuit delay and area metrics from pre-physical synthesis circuit netlists. Once trained, GraPhSyM provides accurate visibility of final design metrics to early EDA stages, such as logic synthesis, without running the slow physical synthesis flow, enabling global co-optimization across stages. Additionally, the swift and precise feedback provided by GraPhSym is instrumental for machine-learning-based EDA optimization frameworks. Given a gate-level netlist of a circuit represented as a graph, GraPhSyM utilizes graph structure, connectivity, and electrical property features to predict the impact of physical synthesis transformations such as buffer insertion and gate sizing. When trained on a dataset of 6000 prefix adder designs synthesized at an aggressive delay target, GraPhSyM can accurately predict the post-synthesis delay (98.3%) and area (96.1%) metrics of unseen adders with a fast 0.22s inference time. Furthermore, we illustrate the compositionality of GraPhSyM by employing the model trained on a fixed delay target to accurately anticipate post-synthesis metrics at a variety of unseen delay targets. Lastly, we report promising generalization capabilities of the GraPhSyM model when it is evaluated on circuits different from the adders it was exclusively trained on. The results show the potential for GraPhSyM to serve as a powerful tool for advanced optimization techniques and as an oracle for EDA machine learning frameworks.
Backscattering reduction in a sharply bent water wave channel

We study theoretically and experimentally how to reduce the backscattering of water waves in a channel with multiple turns. We show that it is not only possible to cancel backscattering but also to achieve a remarkable transmission in such geometries. In order to avoid the reflection that naturally arises at each turn of the waveguide, an anisotropic metamaterial made of closely-spaced thin vertical plates is used. The efficiency of the metamaterial arrangement depends only slightly on the frequency of the incident wave, as long as its wavelength is much larger than the periodicity of the array. This phenomenon is not restricted only to water wave channels but also applies to any type of waves with Neumann boundary conditions.