Miracle-dz/UniFAR-FTUs huggingface dataset:
Causally interpretable meta‐analysis: Clearly defined causal effects and two case studies

Meta‐analysis is commonly used to combine results from multiple clinical trials, but traditional meta‐analysis methods do not refer explicitly to a population of individuals to whom the results apply and it is not clear how to use their results to assess a treatment's effect for a population of interest. We describe recently‐introduced causally interpretable meta‐analysis methods and apply their treatment effect estimators to two individual‐participant data sets. These estimators transport estimated treatment effects from studies in the meta‐analysis to a specified target population using the individuals' potentially effect‐modifying covariates. We consider different regression and weighting methods within this approach and compare the results to traditional aggregated‐data meta‐analysis methods. In our applications, certain versions of the causally interpretable methods performed somewhat better than the traditional methods, but the latter generally did well. The causally interpretable methods offer the most promise when covariates modify treatment effects and our results suggest that traditional methods work well when there is little effect heterogeneity. The causally interpretable approach gives meta‐analysis an appealing theoretical framework by relating an estimator directly to a specific population and lays a solid foundation for future developments.
The effect of observed vertical structure, habits, and size distributions on the solar radiative properties and cloud evolution of cirrus clouds

Cirrus observations taken during EMERALD1 clearly showed vertical structure in ice crystal habit and size distribution. In this study, we use these observations along with radiative and cloud modelling to assess the importance of this observed vertical structure in size and habit on radiative properties and cloud evolution. We show that neglecting the vertical structure results in changes to the solar upwelling and downwelling fluxes of 10 and 8 W m−2, respectively. If further simplifications are made and aggregate and rosettes are treated more simplistically (as columns) then the upwelling and downwelling fluxes are altered by about 12 and 16 W m−2, respectively. [Correction added 8 May 2012 after original online publication: the end of the preceding sentence has been changed from ‘16 and 12 W m−2, respectively’. ] It was noted that the effects of simplifying the vertical structure and habits act in competing ways in terms of the fluxes and therefore emphasize the need to consider both effects in order to improve modelling rather than considering either in isolation.
Doubly-Optimistic Play for Safe Linear Bandits

The safe linear bandit problem (SLB) is an online approach to linear programming with unknown objective and unknown round-wise constraints, under stochastic bandit feedback of rewards and safety risks of actions. We study aggressive \emph{doubly-optimistic play} in SLBs, and their role in avoiding the strong assumptions and poor efficacy associated with extant pessimistic-optimistic solutions. We first elucidate an inherent hardness in SLBs due the lack of knowledge of constraints: there exist `easy' instances, for which suboptimal extreme points have large `gaps', but on which SLB methods must still incur $\Omega(\sqrt{T})$ regret and safety violations due to an inability to refine the location of optimal actions to arbitrary precision. In a positive direction, we propose and analyse a doubly-optimistic confidence-bound based strategy for the safe linear bandit problem, DOSLB, which exploits supreme optimism by using optimistic estimates of both reward and safety risks to select actions. Using a novel dual analysis, we show that despite the lack of knowledge of constraints, DOSLB rarely takes overly risky actions, and obtains tight instance-dependent $O(\log^2 T)$ bounds on both efficacy regret and net safety violations up to any finite precision, thus yielding large efficacy gains at a small safety cost and without strong assumptions. Concretely, we argue that algorithm activates noisy versions of an `optimal' set of constraints at each round, and activation of suboptimal sets of constraints is limited by the larger of a safety and efficacy gap we define.
Frameworks for SNNs: a Review of Data Science-oriented Software and an Expansion of SpykeTorch

Developing effective learning systems for Machine Learning (ML) applications in the Neuromorphic (NM) field requires extensive experimentation and simulation. Software frameworks aid and ease this process by providing a set of ready-to-use tools that researchers can leverage. The recent interest in NM technology has seen the development of several new frameworks that do this, and that add up to the panorama of already existing libraries that belong to neuroscience fields. This work reviews 9 frameworks for the development of Spiking Neural Networks (SNNs) that are specifically oriented towards data science applications. We emphasize the availability of spiking neuron models and learning rules to more easily direct decisions on the most suitable frameworks to carry out different types of research. Furthermore, we present an extension to the SpykeTorch framework that gives users access to a much broader choice of neuron models to embed in SNNs and make the code publicly available.
Utilising the CLT Structure in Stochastic Gradient based Sampling : Improved Analysis and Faster Algorithms

We consider stochastic approximations of sampling algorithms, such as Stochastic Gradient Langevin Dynamics (SGLD) and the Random Batch Method (RBM) for Interacting Particle Dynamcs (IPD). We observe that the noise introduced by the stochastic approximation is nearly Gaussian due to the Central Limit Theorem (CLT) while the driving Brownian motion is exactly Gaussian. We harness this structure to absorb the stochastic approximation error inside the diffusion process, and obtain improved convergence guarantees for these algorithms. For SGLD, we prove the first stable convergence rate in KL divergence without requiring uniform warm start, assuming the target density satisfies a Log-Sobolev Inequality. Our result implies superior first-order oracle complexity compared to prior works, under significantly milder assumptions. We also prove the first guarantees for SGLD under even weaker conditions such as H\"{o}lder smoothness and Poincare Inequality, thus bridging the gap between the state-of-the-art guarantees for LMC and SGLD. Our analysis motivates a new algorithm called covariance correction, which corrects for the additional noise introduced by the stochastic approximation by rescaling the strength of the diffusion. Finally, we apply our techniques to analyze RBM, and significantly improve upon the guarantees in prior works (such as removing exponential dependence on horizon), under minimal assumptions.
Efficient Teacher: Semi-Supervised Object Detection for YOLOv5

Semi-Supervised Object Detection (SSOD) has been successful in improving the performance of both R-CNN series and anchor-free detectors. However, one-stage anchor-based detectors lack the structure to generate high-quality or flexible pseudo labels, leading to serious inconsistency problems in SSOD. In this paper, we propose the Efficient Teacher framework for scalable and effective one-stage anchor-based SSOD training, consisting of Dense Detector, Pseudo Label Assigner, and Epoch Adaptor. Dense Detector is a baseline model that extends RetinaNet with dense sampling techniques inspired by YOLOv5. The Efficient Teacher framework introduces a novel pseudo label assignment mechanism, named Pseudo Label Assigner, which makes more refined use of pseudo labels from Dense Detector. Epoch Adaptor is a method that enables a stable and efficient end-to-end semi-supervised training schedule for Dense Detector. The Pseudo Label Assigner prevents the occurrence of bias caused by a large number of low-quality pseudo labels that may interfere with the Dense Detector during the student-teacher mutual learning mechanism, and the Epoch Adaptor utilizes domain and distribution adaptation to allow Dense Detector to learn globally distributed consistent features, making the training independent of the proportion of labeled data. Our experiments show that the Efficient Teacher framework achieves state-of-the-art results on VOC, COCO-standard, and COCO-additional using fewer FLOPs than previous methods. To the best of our knowledge, this is the first attempt to apply Semi-Supervised Object Detection to YOLOv5.Code is available: https://github.com/AlibabaResearch/efficientteacher
The formation of nuclei in newly formed galaxies and the evolution of the quasar population

A model to explain the evolution of the quasar luminosity function is presented. At intermediate and high redshifts, the formation of'new'black holes at each step in the hierarchical growth of structure is invoked as the driving force of the strong evolution of the quasar population. Quasars are assumed to be short-lived, constituting the first phase of the formation of a galaxy in the potential well of a dark matter halo. The timelag between halo virialization and the birth of the quasar is expected to be short compared to the cosmological time-scale, even at high redshifts. Simple assumptions are made to relate the luminosity of a quasar to the mass of its central black hole and to the mass of its corresponding host object
AN ENERGY-TYPE INEQUALITY

In this paper we will prove an Energy-Type inequality for the mild solution of the linear evolution equation. And for showing the strength and power of this inequality we will use this inequality to show continuity of the solution with respect to a parameter of the semilinear evolution equation with monotone nonlinearity. Mathematics subject classification (1991): 34G10, 45N05.
The Capacity for Moral Self-Correction in Large Language Models

We test the hypothesis that language models trained with reinforcement learning from human feedback (RLHF) have the capability to"morally self-correct"-- to avoid producing harmful outputs -- if instructed to do so. We find strong evidence in support of this hypothesis across three different experiments, each of which reveal different facets of moral self-correction. We find that the capability for moral self-correction emerges at 22B model parameters, and typically improves with increasing model size and RLHF training. We believe that at this level of scale, language models obtain two capabilities that they can use for moral self-correction: (1) they can follow instructions and (2) they can learn complex normative concepts of harm like stereotyping, bias, and discrimination. As such, they can follow instructions to avoid certain kinds of morally harmful outputs. We believe our results are cause for cautious optimism regarding the ability to train language models to abide by ethical principles.
Cliff-Learning

We study the data-scaling of transfer learning from foundation models in the low-downstream-data regime. We observe an intriguing phenomenon which we call cliff-learning. Cliff-learning refers to regions of data-scaling laws where performance improves at a faster than power law rate (i.e. regions of concavity on a log-log scaling plot). We conduct an in-depth investigation of foundation-model cliff-learning and study toy models of the phenomenon. We observe that the degree of cliff-learning reflects the degree of compatibility between the priors of a learning algorithm and the task being learned.
Multi-graded Featherweight Java

Resource-aware type systems statically approximate not only the expected result type of a program, but also the way external resources are used, e.g., how many times the value of a variable is needed. We extend the type system of Featherweight Java to be resource-aware, parametrically on an arbitrary grade algebra modeling a specific usage of resources. We prove that this type system is sound with respect to a resource-aware version of reduction, that is, a well-typed program has a reduction sequence which does not get stuck due to resource consumption. Moreover, we show that the available grades can be heterogeneous, that is, obtained by combining grades of different kinds, via a minimal collection of homomorphisms from one kind to another. Finally, we show how grade algebras and homomorphisms can be specified as Java classes, so that grade annotations in types can be written in the language itself.
Efficient Inversion of Matrix φ-Functions of Low Order

The paper is concerned with efficient numerical methods for solving a linear system $\phi(A) x= b$, where $\phi(z)$ is a $\phi$-function and $A\in \mathbb R^{N\times N}$. In particular in this work we are interested in the computation of ${\phi(A)}^{-1} b$ for the case where $\phi(z)=\phi_1(z)=\displaystyle\frac{e^z-1}{z}, \quad \phi(z)=\phi_2(z)=\displaystyle\frac{e^z-1-z}{z^2}$. Under suitable conditions on the spectrum of $A$ we design fast algorithms for computing both ${\phi_\ell(A)}^{-1}$ and ${\phi_\ell(A)}^{-1} b$ based on Newton's iteration and Krylov-type methods, respectively. Adaptations of these schemes for structured matrices are considered. In particular the cases of banded and more generally quasiseparable matrices are investigated. Numerical results are presented to show the effectiveness of our proposed algorithms.
Team Triple-Check at Factify 2: Parameter-Efficient Large Foundation Models with Feature Representations for Multi-Modal Fact Verification

Multi-modal fact verification has become an important but challenging issue on social media due to the mismatch between the text and images in the misinformation of news content, which has been addressed by considering cross-modalities to identify the veracity of the news in recent years. In this paper, we propose the Pre-CoFactv2 framework with new parameter-efficient foundation models for modeling fine-grained text and input embeddings with lightening parameters, multi-modal multi-type fusion for not only capturing relations for the same and different modalities but also for different types (i.e., claim and document), and feature representations for explicitly providing metadata for each sample. In addition, we introduce a unified ensemble method to boost model performance by adjusting the importance of each trained model with not only the weights but also the powers. Extensive experiments show that Pre-CoFactv2 outperforms Pre-CoFact by a large margin and achieved new state-of-the-art results at the Factify challenge at AAAI 2023. We further illustrate model variations to verify the relative contributions of different components. Our team won the first prize (F1-score: 81.82%) and we made our code publicly available at https://github.com/wwweiwei/Pre-CoFactv2-AAAI-2023.
Measuring the Instability of Fine-Tuning

Fine-tuning pre-trained language models on downstream tasks with varying random seeds has been shown to be unstable, especially on small datasets. Many previous studies have investigated this instability and proposed methods to mitigate it. However, most of these studies only used the standard deviation of performance scores (SD) as their measure, which is a narrow characterization of instability. In this paper, we analyze SD and six other measures quantifying instability of different granularity levels. Moreover, we propose a systematic evaluation framework of these measures’ validity. Finally, we analyze the consistency and difference between different measures by reassessing existing instability mitigation methods. We hope our results will inform better measurements of the fine-tuning instability.
The 2022 High-energy Outburst and Radio Disappearing Act of the Magnetar 1E 1547.0–5408

We report the radio and high-energy properties of a new outburst from the radio-loud magnetar 1E 1547.0−5408. Following the detection of a short burst from the source with Swift-BAT on 2022 April 7, observations by NICER detected an increased flux peaking at (6.0 ± 0.4) × 10−11 erg s−1 cm−2 in the soft X-ray band, falling to a baseline level of 1.7 × 10−11 erg s−1 cm−2 over a 17 day period. Joint spectroscopic measurements by NICER and NuSTAR indicated no change in the hard nonthermal tail despite the prominent increase in soft X-rays. Observations at radio wavelengths with Murriyang, the 64 m Parkes radio telescope, revealed that the persistent radio emission from the magnetar disappeared at least 22 days prior to the initial Swift-BAT detection and was redetected two weeks later. Such behavior is unprecedented in a radio-loud magnetar, and may point to an unnoticed slow rise in the high-energy activity prior to the detected short bursts. Finally, our combined radio and X-ray timing revealed the outburst coincided with a spin-up glitch, where the spin frequency and spin-down rate increased by 0.2 ± 0.1 μHz and (−2.4 ± 0.1) × 10−12 s−2, respectively. A linear increase in the spin-down rate of (−2.0 ± 0.1) × 10−19 s−3 was also observed over 147 days of postoutburst timing. Our results suggest that the outburst may have been associated with a reconfiguration of the quasi-polar field lines, likely signaling a changing twist, accompanied by spatially broader heating of the surface and a brief quenching of the radio signal, yet without any measurable impact on the hard X-ray properties.
AI Chat Assistants can Improve Conversations about Divisive Topics

A rapidly increasing amount of human conversation occurs online. But divisiveness and conflict can fester in text-based interactions on social media platforms, in messaging apps, and on other digital forums. Such toxicity increases polarization and, importantly, corrodes the capacity of diverse societies to develop efficient solutions to complex social problems that impact everyone [1, 2]. Scholars and civil society groups promote interventions that can make interpersonal conversations less divisive or more productive in offline settings [3–5], but scaling these efforts to the amount of discourse that occurs online is extremely challenging. We present results of a large-scale experiment that demonstrates how online conversations about divisive topics can be improved with artificial intelligence tools. Specifically, we employ a large language model to make real-time, evidence-based recommendations intended to improve participants’ perception of feeling understood in conversations. We find that these interventions improve the reported quality of the conversation, reduce political divisiveness, and improve the tone, without systematically changing the content of the conversation or moving people’s policy attitudes. These findings have important implications for future research on social media, political deliberation, and the growing community of scholars interested in the place of artificial intelligence within computational social science.
Advancing Software Citation Implementation (Software Citation Workshop 2022)

Software is foundationally important to scientific and social progress, however, traditional acknowledgment of the use of others' work has not adapted in step with the rapid development and use of software in research. This report outlines a series of collaborative discussions that brought together an international group of stakeholders and experts representing many communities, forms of labor, and expertise. Participants addressed specific challenges about software citation that have so far gone unresolved. The discussions took place in summer 2022 both online and in-person and involved a total of 51 participants. The activities described in this paper were intended to identify and prioritize specific software citation problems, develop (potential) interventions, and lay out a series of mutually supporting approaches to address them. The outcomes of this report will be useful for the GLAM (Galleries, Libraries, Archives, Museums) community, repository managers and curators, research software developers, and publishers.
A study on angular distribution of THz radiation driven by two-colour laser-induced microplasma

We present an analytical method to study the angular distribution of THz radiation driven by two-colour laser-induced microplasma in nitrogen gas. Directionally dependent temporal waveforms of the radiated THz pulse is calculated. To do so, we have extended the photo-current model to calculate three-dimensional distribution of the photo-induced current from which the far-field THz radiation is computed. We have also studied the effect of optical polarization state and phase difference on the angular distribution of radiation. Our study shows that for two orthogonally-polarized colours, not only THz polarization state, but also the angular distribution of THz radiation is mostly influenced by the second optical harmonic. The possibility of rotating the angular radiation pattern around the axis of the laser propagation is illustrated for circularly-polarized laser drivers by changing the optical phase difference.
Data Forensics in Diffusion Models: A Systematic Analysis of Membership Privacy

In recent years, diffusion models have achieved tremendous success in the field of image generation, becoming the stateof-the-art technology for AI-based image processing applications. Despite the numerous benefits brought by recent advances in diffusion models, there are also concerns about their potential misuse, specifically in terms of privacy breaches and intellectual property infringement. In particular, some of their unique characteristics open up new attack surfaces when considering the real-world deployment of such models. With a thorough investigation of the attack vectors, we develop a systematic analysis of membership inference attacks on diffusion models and propose novel attack methods tailored to each attack scenario specifically relevant to diffusion models. Our approach exploits easily obtainable quantities and is highly effective, achieving near-perfect attack performance (>0.9 AUCROC) in realistic scenarios. Our extensive experiments demonstrate the effectiveness of our method, highlighting the importance of considering privacy and intellectual property risks when using diffusion models in image generation tasks.
Deep Learning for Detection and Localization of B-Lines in Lung Ultrasound

Lung ultrasound (LUS) is an important imaging modality used by emergency physicians to assess pulmonary congestion at the patient bedside. B-line artifacts in LUS videos are key findings associated with pulmonary congestion. Not only can the interpretation of LUS be challenging for novice operators, but visual quantification of B-lines remains subject to observer variability. In this work, we investigate the strengths and weaknesses of multiple deep learning approaches for automated B-line detection and localization in LUS videos. We curate and publish, BEDLUS, a new ultrasound dataset comprising 1,419 videos from 113 patients with a total of 15,755 expert-annotated B-lines. Based on this dataset, we present a benchmark of established deep learning methods applied to the task of B-line detection. To pave the way for interpretable quantification of B-lines, we propose a novel “single-point” approach to B-line localization using only the point of origin. Our results show that (a) the area under the receiver operating characteristic curve ranges from 0.864 to 0.955 for the benchmarked detection methods, (b) within this range, the best performance is achieved by models that leverage multiple successive frames as input, and (c) the proposed single-point approach for B-line localization reaches an F$_{1}$-score of 0.65, performing on par with the inter-observer agreement. The dataset and developed methods can facilitate further biomedical research on automated interpretation of lung ultrasound with the potential to expand the clinical utility.