Miracle-dz/UniFAR-FTUs huggingface dataset:
Jorge: Approximate Preconditioning for GPU-efficient Second-order Optimization

Despite their better convergence properties compared to first-order optimizers, second-order optimizers for deep learning have been less popular due to their significant computational costs. The primary efficiency bottleneck in such optimizers is matrix inverse calculations in the preconditioning step, which are expensive to compute on GPUs. In this paper, we introduce Jorge, a second-order optimizer that promises the best of both worlds -- rapid convergence benefits of second-order methods, and high computational efficiency typical of first-order methods. We address the primary computational bottleneck of computing matrix inverses by completely eliminating them using an approximation of the preconditioner computation. This makes Jorge extremely efficient on GPUs in terms of wall-clock time. Further, we describe an approach to determine Jorge's hyperparameters directly from a well-tuned SGD baseline, thereby significantly minimizing tuning efforts. Our empirical evaluations demonstrate the distinct advantages of using Jorge, outperforming state-of-the-art optimizers such as SGD, AdamW, and Shampoo across multiple deep learning models, both in terms of sample efficiency and wall-clock time.
Model Merging by Uncertainty-Based Gradient Matching

Models trained on different datasets can be merged by a weighted-averaging of their parameters, but why does it work and when can it fail? Here, we connect the inaccuracy of weighted-averaging to mismatches in the gradients and propose a new uncertainty-based scheme to improve the performance by reducing the mismatch. The connection also reveals implicit assumptions in other schemes such as averaging, task arithmetic, and Fisher-weighted averaging. Our new method gives consistent improvements for large language models and vision transformers, both in terms of performance and robustness to hyperparameters.
Interpreting Sequence-Levenshtein distance for determining error type and frequency between two embedded sequences of equal length

Levenshtein distance is a commonly used edit distance metric, typically applied in language processing, and to a lesser extent, in molecular biology analysis. Biological nucleic acid sequences are often embedded in longer sequences and are subject to insertion and deletion errors that introduce frameshift during sequencing. These frameshift errors are due to string context and should not be counted as true biological errors. Sequence-Levenshtein distance is a modification to Levenshtein distance that is permissive of frameshift error without additional penalty. However, in a biological context Levenshtein distance needs to accommodate both frameshift and weighted errors, which Sequence-Levenshtein distance cannot do. Errors are weighted when they are associated with a numerical cost that corresponds to their frequency of appearance. Here, we describe a modification that allows the use of Levenshtein distance and Sequence-Levenshtein distance to appropriately accommodate penalty-free frameshift between embedded sequences and correctly weight specific error types.
Equipping Federated Graph Neural Networks with Structure-aware Group Fairness

Graph Neural Networks (GNNs) are used for graph data processing across various domains. Centralized training of GNNs often faces challenges due to privacy and regulatory issues, making federated learning (FL) a preferred solution in a distributed paradigm. However, GNNs may inherit biases from training data, causing these biases to propagate to the global model in distributed scenarios. To address this issue, we introduce $\mathrm{F}^{2}$GNN, a Fair Federated Graph Neural Network, to enhance group fairness. Recognizing that bias originates from both data and algorithms, $\mathrm{F}^{2}$GNN aims to mitigate both types of bias under federated settings. We offer theoretical insights into the relationship between data bias and statistical fairness metrics in GNNs. Building on our theoretical analysis, $\mathrm{F}^{2}$GNN features a fairness-aware local model update scheme and a fairness-weighted global model update scheme, considering both data bias and local model fairness during aggregation. Empirical evaluations show $\mathrm{F}^{2}$GNN outperforms SOTA baselines in fairness and accuracy.
A Systematic Study of Performance Disparities in Multilingual Task-Oriented Dialogue Systems

Achieving robust language technologies that can perform well across the world's many languages is a central goal of multilingual NLP. In this work, we take stock of and empirically analyse task performance disparities that exist between multilingual task-oriented dialogue (ToD) systems. We first define new quantitative measures of absolute and relative equivalence in system performance, capturing disparities across languages and within individual languages. Through a series of controlled experiments, we demonstrate that performance disparities depend on a number of factors: the nature of the ToD task at hand, the underlying pretrained language model, the target language, and the amount of ToD annotated data. We empirically prove the existence of the adaptation and intrinsic biases in current ToD systems: e.g., ToD systems trained for Arabic or Turkish using annotated ToD data fully parallel to English ToD data still exhibit diminished ToD task performance. Beyond providing a series of insights into the performance disparities of ToD systems in different languages, our analyses offer practical tips on how to approach ToD data collection and system development for new languages.
Tight Short-Lived Signatures

A Time-lock puzzle (TLP) sends information into the future: a predetermined number of sequential computations must occur (i.e., a predetermined amount of time must pass) to retrieve the information, regardless of parallelization. Buoyed by the excitement around secure decentralized applications and cryptocurrencies, the last decade has witnessed numerous constructions of TLP variants and related applications (e.g., cost-efficient blockchain designs, randomness beacons, e-voting, etc.). In this poster, we first extend the notion of TLP by formally defining the"time-lock public key encryption"(TLPKE) scheme. Next, we introduce and construct a"tight short-lived signatures"scheme using our TLPKE. Furthermore, to test the validity of our proposed schemes, we do a proof-of-concept implementation and run detailed simulations.
The Locality and Symmetry of Positional Encodings

Positional Encodings (PEs) are used to inject word-order information into transformer-based language models. While they can significantly enhance the quality of sentence representations, their specific contribution to language models is not fully understood, especially given recent findings that various positional encodings are insensitive to word order. In this work, we conduct a systematic study of positional encodings in \textbf{Bidirectional Masked Language Models} (BERT-style) , which complements existing work in three aspects: (1) We uncover the core function of PEs by identifying two common properties, Locality and Symmetry; (2) We show that the two properties are closely correlated with the performances of downstream tasks; (3) We quantify the weakness of current PEs by introducing two new probing tasks, on which current PEs perform poorly. We believe that these results are the basis for developing better PEs for transformer-based language models. The code is available at \faGithub~ \url{https://github.com/tigerchen52/locality\_symmetry}
Conservative Inference for Counterfactuals

In causal inference, the joint law of a set of counterfactual random variables is generally not identified. We show that a conservative version of the joint law - corresponding to the smallest treatment effect - is identified. Finding this law uses recent results from optimal transport theory. Under this conservative law we can bound causal effects and we may construct inferences for each individual's counterfactual dose-response curve. Intuitively, this is the flattest counterfactual curve for each subject that is consistent with the distribution of the observables. If the outcome is univariate then, under mild conditions, this curve is simply the quantile function of the counterfactual distribution that passes through the observed point. This curve corresponds to a nonparametric rank preserving structural model.
Eureka: Human-Level Reward Design via Coding Large Language Models

Large Language Models (LLMs) have excelled as high-level semantic planners for sequential decision-making tasks. However, harnessing them to learn complex low-level manipulation tasks, such as dexterous pen spinning, remains an open problem. We bridge this fundamental gap and present Eureka, a human-level reward design algorithm powered by LLMs. Eureka exploits the remarkable zero-shot generation, code-writing, and in-context improvement capabilities of state-of-the-art LLMs, such as GPT-4, to perform evolutionary optimization over reward code. The resulting rewards can then be used to acquire complex skills via reinforcement learning. Without any task-specific prompting or pre-defined reward templates, Eureka generates reward functions that outperform expert human-engineered rewards. In a diverse suite of 29 open-source RL environments that include 10 distinct robot morphologies, Eureka outperforms human experts on 83% of the tasks, leading to an average normalized improvement of 52%. The generality of Eureka also enables a new gradient-free in-context learning approach to reinforcement learning from human feedback (RLHF), readily incorporating human inputs to improve the quality and the safety of the generated rewards without model updating. Finally, using Eureka rewards in a curriculum learning setting, we demonstrate for the first time, a simulated Shadow Hand capable of performing pen spinning tricks, adeptly manipulating a pen in circles at rapid speed.
Optimizing a Broad Energy High Purity Germanium (BEGe) Detector Operated at Shallow Depth in Abu Dhabi

In this work we present the characterization of a Broad Energy Germanium (BEGe) type High Purity Germanium (HPGe) detector, with a carbon fiber entrance window thickness of 0.6 mm and an active area of 6305 mm2, operated at shallow depth (~ 8m) in Abu Dhabi, UAE. A 1.6 keV Full Width Half Maximum (FWHM) was obtained for the 662 keV peak of 137Cs. A muon veto was applied, reducing the background by 8 % (for energies greater than 100 keV). Flushing the volume around the detector endcap with nitrogen gas, to remove radon and thus its progeny, further reduced the background by ~3 %. A thorough analysis for the shaping filter parameters showed that the detector has better resolution at low rise-time values (2 - 5 us) especially for low energy gamma (<600keV), keeping the flattop value fixed at 1.1 us.
Loop Copilot: Conducting AI Ensembles for Music Generation and Iterative Editing

Creating music is iterative, requiring varied methods at each stage. However, existing AI music systems fall short in orchestrating multiple subsystems for diverse needs. To address this gap, we introduce Loop Copilot, a novel system that enables users to generate and iteratively refine music through an interactive, multi-round dialogue interface. The system uses a large language model to interpret user intentions and select appropriate AI models for task execution. Each backend model is specialized for a specific task, and their outputs are aggregated to meet the user's requirements. To ensure musical coherence, essential attributes are maintained in a centralized table. We evaluate the effectiveness of the proposed system through semi-structured interviews and questionnaires, highlighting its utility not only in facilitating music creation but also its potential for broader applications.
The collapse of eastern Bering Sea snow crab

The snow crab is an iconic species in the Bering Sea that supports an economically important fishery and undergoes extensive monitoring and management. Since 2018, more than 10 billion snow crab have disappeared from the eastern Bering Sea, and the population collapsed to historical lows in 2021. We link this collapse to a marine heatwave in the eastern Bering Sea during 2018 and 2019. Calculated caloric requirements, reduced spatial distribution, and observed body conditions suggest that starvation played a role in the collapse. The mortality event appears to be one of the largest reported losses of motile marine macrofauna to marine heatwaves globally. Description Editor’s summary Marine heatwaves, a component of our impact on the Earth’s climate, can bring both expected and unexpected environmental change. Between 2018 and 2021, after a period of historically high crab abundance and a series of marine heatwaves, the population of snow crab in the Bering Sea declined by 10 billion. Szuwalksi et al. used survey data to model the potential drivers of the decline in this ecologically and commercially important species. They found that the temperature of the water was not above the species’ thermal limits, but it did increase their caloric needs considerably (see the Perspective by Kruse). This increase, in conjunction with a restriction in range, led to an unexpected mass starvation event. —Sacha Vignieri A marine heatwave precipitated the collapse of the eastern Bering Sea snow crab population.
Flow Chamber Analyses in Cardiovascular Research: Impact of Platelets and the Intercellular Crosstalk with Endothelial Cells, Leukocytes, and Red Blood Cells

Abstract Platelets are main drivers of thrombus formation. Besides platelet aggregate formation, platelets interact with different blood cells such as red blood and white blood cells (RBCs, WBCs) and endothelial cells (ECs), to promote thrombus formation and inflammation. In the past, the role of different proteins in platelet adhesion, activation, and aggregate formation has been analyzed using platelets/mice with a genetic loss of a certain protein. These knock-out mouse models have been investigated for changes in experimental arterial thrombosis or hemostasis. In this review, we focused on the Maastricht flow chamber, which is a very elegant tool to analyze thrombus formation under flow using whole blood or different blood cell components of genetically modified mice. Besides, the interaction of platelets with RBCs, WBCs, and ECs under flow conditions has been evaluated with regard to thrombus formation and platelet-mediated inflammation. Importantly, alterations in thrombus formation as emerged in the flow chamber frequently reflect arterial thrombosis in different mouse models. Thus, the results of flow chamber experiments in vitro are excellent indicators for differences in arterial thrombosis in vivo. Taken together, the Maastricht flow chamber can be used to (1) determine the severity of platelet alterations in different knock-out mice; (2) analyze differences in platelet adhesion, aggregation, and activation; (3) investigate collagen and non–collagen-dependent alterations of thrombus formation; and (4) highlight differences in the interaction of platelets with different blood/ECs. Thus, this experimental approach is a useful tool to increase our understanding of signaling mechanisms that drive arterial thrombosis and hemostasis.
Land reforms and impact on land use in the uplands of Vietnam and Laos: Environmental protection or poverty alleviation? Les réformes foncières dans les montagnes du Vietnam et du Laos : protection de l'environnement ou lutte contre la pauvreté ?

Environmental protection and poverty alleviation are central to sustainable development as stated in the Millennium declaration. For many States, land reform is a key leverage point in converting farming systems into new ones that better address the new priorities. In the 1990s, the Governments of Vietnam and Laos launched land reforms aimed at: (1) entrusting the village communities with protecting the forests; and (2) prompting the families to invest more in intensive farming as a result of more secure land tenure. It is not by chance that similar policies appeared synchronously in the two countries, as the two neighbouring States share a common history from French colonisation to market socialism, following a period of planned and collectivist economy. Both countries were supported by foreign development agencies in implementing their land reforms. The policy was the result of a timely convergence of goals between groups with diverging interests, i.e. (i) the socialist model implying control over agricultural production and upland minorities, (ii) the free-market model from the international development banks, (iii) the scientific forestry model from industrialised countries and (iv) the conservationist lobby of the environmental organisations. Nevertheless, there are considerable differences in the agricultural situation both within and between the two countries, and these differences explain the contrasting results of national and monolithic policies. Two case studies carried out in neighbouring but different upland regions show how local communities reacted to the land reforms. Comparison with other case studies in Vietnam and Laos revealed that the land reforms have had mixed effects both in terms of preservation of natural resources and of poverty alleviation. Is it possible to reconcile these two goals? This paper contributes to the debate by bringing new insights into local adaptations in the face of global policies.
Handgrip strength as a screening tool for diabetes in resource-constrained settings: a potential solution to overcome barriers to diagnosis

Abstract Background Information Diabetes mellitus is an escalating global health concern, especially in low and middle-income countries. The high cost and inaccessibility of diagnostic tools in resource-constrained settings have heightened the need for alternative screening methods. Handgrip strength (HGS), a measure of muscle strength, emerges as a potential non-invasive and affordable screening tool for diabetes, particularly in areas with limited healthcare access. Objective To investigate the relationship between handgrip strength and blood glucose regulation in non-diabetic young adults and to provide valuable insights into the potential of handgrip strength as a preventive and affordable approach to managing diabetes. Methods A cross-sectional study was conducted involving 59 students (aged 18-21) from the University of Ilorin, Nigeria. Handgrip strength was measured using a dynamometer, and its relationship with blood glucose regulatory markers, such as fasting blood glucose, 2-hour post-prandial glucose, and HbA1c, was analyzed. Multiple regression models were utilized to examine the potential associations. Results Findings revealed significant associations between HGS and glucose regulation markers, particularly FBS, among males. In females, the relationship was evident only after adjusting for body mass index (BMI). Furthermore, a notable relationship between HGS and 2-hour post-prandial glucose levels was observed in females but not in males. However, no significant associations were found between HGS and serum insulin levels across genders. Conclusion Our study introduces HGS as a practical and cost-effective screening tool for blood glucose regulation disorders, aligning with existing literature and offering a personalized approach to management. In resource-constrained settings, HGS becomes significant, addressing diagnostic barriers and potentially revolutionizing diabetes management. However, limitations include a small sample size of 59 students and restrictions to specific demographics, emphasizing the need for future studies in diverse populations to validate HGS's efficacy in real-world, resource-constrained settings.
Social Determinants Predicting the Community Pharmacists’ Workforce Preparedness for, and Response to, the Public Health Emergencies

Abstract Objectives: To identify the predicting factors that contribute to preparedness for public health emergencies among community pharmacists in India. Methods: Multistage cluster sampling was done. The geographic breakdown was done based on villages and areas and used as clusters. A simple random method was done in the first stage to select the villages as clusters. From each selected village, a simple random method was done in the second stage to select the areas. From each selected area, all the community pharmacies were selected. The survey questionnaire had 3 sections with 43 items: (A) demographic information, (B) preparedness, (C) response toward infectious diseases. The participants chose “Yes/No”, in sections B and C. A score of 1 was given for “Yes”, and a score of zero was given for “No”. Results: Multiple correlation analyses were conducted between participants’ preparedness and response (PR) scores and independent variables. The independent variables such as “More than one Pharmacist working in a pharmacy”, “Pharmacists who are trained more than once on disaster management”, and encountered more than 1 patient with the infectious disease were positively and significantly correlated with the dependent variable (PR scores). Conclusions: Community pharmacists were aware of the issues they may face in their community concerning public health emergencies. They believed that the medications available in their pharmacy are sufficient to face any emergency. They could identify the clinical manifestations of public health emergency conditions and provide counselling to the customers toward them. Community pharmacists who were trained more than once in disaster management were the strongest predicting factor.
Additions to the knowledge of South Indian Scaphisomatini (Coleoptera: Staphylinidae: Scaphidiinae)

Abstract: A study of a collection of South Indian Scaphidiinae yielded six new species and several interesting records. The following species are described: Scaphisoma boukali sp. nov., S. divali sp. nov., S. holi sp. nov., S. kejvali sp. nov., S. onam sp. nov. and S. pooram sp. nov. Scaphobaeocera stipes Löbl, 1971 is reported from India for the first time. An updated key to identification of South Indian species of Scaphisoma is provided.
Computer Vision-Based Algorithms for Recognition of Construction and Demolition Waste Materials

The construction industry generates a significant amount of waste, posing challenges for efficient waste management and resource recovery. This paper presents a preliminary study on the use of lightweight computer vision (CV) algorithms for the automatic recognition of construction and demolition waste (CDW) materials. Utilizing image datasets acquired by drones, the study aims to develop strategies for distinguishing between individual CDW materials based on the mean intensity gradient, brightness, and relative representation of color channels. Results indicate that the proposed method can effectively recognize crucial CDW materials, paving the way for potential applications in industry and geodesy. Further research is needed to test additional materials and metrics to refine the method for practical implementation.