Miracle-dz/UniFAR-FTUs huggingface dataset:
AdaMix: Mixture-of-Adapter for Parameter-efficient Tuning of Large Language Models

Fine-tuning large-scale pre-trained language models to downstream tasks require updating hundreds of millions of parameters. This not only increases the serving cost to store a large copy of the model weights for every task, but also exhibits instability during few-shot task adaptation. Parameter-efficient techniques have been developed that tune small trainable components (e
ORCA: Interpreting Prompted Language Models via Locating Supporting Data Evidence in the Ocean of Pretraining Data

Large pretrained language models have been performing increasingly well in a variety of downstream tasks via prompting. However, it remains unclear from where the model learns the task-specific knowledge, especially in a zero-shot setup. In this work, we want to find evidence of the model's task-specific competence from pretraining and are specifically interested in locating a very small subset of pretraining data that directly supports the model in the task. We call such a subset supporting data evidence and propose a novel method ORCA to effectively identify it, by iteratively using gradient information related to the downstream task. This supporting data evidence offers interesting insights about the prompted language models: in the tasks of sentiment analysis and textual entailment, BERT shows a substantial reliance on BookCorpus, the smaller corpus of BERT's two pretraining corpora, as well as on pretraining examples that mask out synonyms to the task verbalizers.
Detecting Label Errors by Using Pre-Trained Language Models

We show that large pre-trained language models are inherently highly capable of identifying label errors in natural language datasets: simply examining out-of-sample data points in descending order of fine-tuned task loss significantly outperforms more complex error-detection mechanisms proposed in previous work. To this end, we contribute a novel method for introducing realistic, human-originated label noise into existing crowdsourced datasets such as SNLI and TweetNLP. We show that this noise has similar properties to real, hand-verified label errors, and is harder to detect than existing synthetic noise, creating challenges for model robustness.We argue that human-originated noise is a better standard for evaluation than synthetic noise. Finally, we use crowdsourced verification to evaluate the detection of real errors on IMDB, Amazon Reviews, and Recon, and confirm that pre-trained models perform at a 9–36% higher absolute Area Under the Precision-Recall Curve than existing models.
A Continuum Model of Stable Matching with Finite Capacities

This paper introduces a unified framework for stable matching, which nests the traditional definition of stable matching in finite markets and the continuum definition of stable matching from Azevedo and Leshno (2016) as special cases. Within this framework, I identify a novel continuum model, which makes individual-level probabilistic predictions. This new model always has a unique stable outcome, which can be found using an analog of the Deferred Acceptance algorithm. The crucial difference between this model and that of Azevedo and Leshno (2016) is that they assume that the amount of student interest at each school is deterministic, whereas my proposed alternative assumes that it follows a Poisson distribution. As a result, this new model accurately predicts the simulated distribution of cutoffs, even for markets with only ten schools and twenty students. This model generates new insights about the number and quality of matches. When schools are homogeneous, it provides upper and lower bounds on students' average rank, which match results from Ashlagi, Kanoria and Leshno (2017) but apply to more general settings. This model also provides clean analytical expressions for the number of matches in a platform pricing setting considered by Marx and Schummer (2021).
DisinfoMeme: A Multimodal Dataset for Detecting Meme Intentionally Spreading Out Disinformation

Disinformation has become a serious problem on social media. In particular, given their short format, visual attraction, and humorous nature, memes have a significant advantage in dissemination among online communities, making them an effective vehicle for the spread of disinformation. We present DisinfoMeme to help detect disinformation memes. The dataset contains memes mined from Reddit covering three current topics: the COVID-19 pandemic, the Black Lives Matter movement, and veganism/vegetarianism. The dataset poses multiple unique challenges: limited data and label imbalance, reliance on external knowledge, multimodal reasoning, layout dependency, and noise from OCR. We test multiple widely-used unimodal and multimodal models on this dataset. The experiments show that the room for improvement is still huge for current models.
Leveraging Locality in Abstractive Text Summarization

Neural attention models have achieved significant improvements on many natural language processing tasks. However, the quadratic memory complexity of the self-attention module with respect to the input length hinders their applications in long text summarization. Instead of designing more efficient attention modules, we approach this problem by investigating if models with a restricted context can have competitive performance compared with the memory-efficient attention models that maintain a global context by treating the input as a single sequence. Our model is applied to individual pages, which contain parts of inputs grouped by the principle of locality, during both the encoding and decoding stages. We empirically investigated three kinds of locality in text summarization at different levels of granularity, ranging from sentences to documents. Our experimental results show that our model has a better performance compared with strong baseline models with efficient attention modules, and our analysis provides further insights into our locality-aware modeling strategy.
Over-the-Air Design of GAN Training for mmWave MIMO Channel Estimation

Future wireless systems are trending towards higher carrier frequencies that offer larger communication bandwidth but necessitate the use of large antenna arrays. Signal processing techniques for channel estimation currently deployed in wireless devices do not scale well to this “high-dimensional” regime in terms of performance and pilot overhead. Meanwhile, training deep learning-based approaches for channel estimation requires large labeled datasets mapping pilot measurements to clean channel realizations, which can only be generated offline using simulated channels. In this paper, we develop a novel unsupervised over-the-air (OTA) algorithm that utilizes noisy received pilot measurements to train a deep generative model to output beamspace MIMO channel realizations. Our approach leverages Generative Adversarial Networks (GAN), while using a conditional input to distinguish between Line-of-Sight (LOS) and Non-Line-of-Sight (NLOS) channel realizations. We also present a federated implementation of the OTA algorithm that distributes the GAN training over multiple users and greatly reduces the user side computation. We then formulate channel estimation from a limited number of pilot measurements as an inverse problem and reconstruct the channel by optimizing the input vector of the trained generative model. Our proposed approach significantly outperforms Orthogonal Matching Pursuit on both LOS and NLOS channel models, and EM-GM-AMP – an Approximate Message Passing algorithm – on LOS channel models, while achieving comparable performance on NLOS channel models in terms of the normalized channel reconstruction error. More importantly, our proposed framework has the potential to be trained online using real noisy pilot measurements, is not restricted to a specific channel model and can even be utilized for a federated OTA design of a dataset generator from noisy data.
Cross-Domain Style Mixing for Face Cartoonization

Cartoon domain has recently gained increasing popular-ity. Previous studies have attempted quality portrait stylization into the cartoon domain; however, this poses a great challenge since they have not properly addressed the critical constraints, such as requiring a large number of training images or the lack of support for abstract cartoon faces. Recently, a layer swapping method has been used for stylization requiring only a limited number of training images; however, its use cases are still narrow as it inherits the re-maining issues. In this paper, we propose a novel method called Cross-domain Style mixing, which combines two latent codes from two different domains. Our method effectively stylizes faces into multiple cartoon characters at various face abstraction levels using only a single generator without even using a large number of training images.
The anisotropic min‐max theory: Existence of anisotropic minimal and CMC surfaces

We prove the existence of nontrivial closed surfaces with constant anisotropic mean curvature with respect to elliptic integrands in closed smooth 3–dimensional Riemannian manifolds. The constructed min‐max surfaces are smooth with at most one singular point. The constant anisotropic mean curvature can be fixed to be any real number. In particular, we partially solve a conjecture of Allard in dimension 3.
Accelerating Diffusion Models via Early Stop of the Diffusion Process

Denoising Diffusion Probabilistic Models (DDPMs) have achieved impressive performance on various generation tasks. By modeling the reverse process of gradually diffusing the data distribution into a Gaussian distribution, generating a sample in DDPMs can be regarded as iteratively denoising a randomly sampled Gaussian noise. However, in practice DDPMs often need hundreds even thousands of denoising steps to obtain a high-quality sample from the Gaussian noise, leading to extremely low inference efficiency. In this work, we propose a principled acceleration strategy, referred to as Early-Stopped DDPM (ES-DDPM), for DDPMs. The key idea is to stop the diffusion process early where only the few initial diffusing steps are considered and the reverse denoising process starts from a non-Gaussian distribution. By further adopting a powerful pre-trained generative model, such as GAN and VAE, in ES-DDPM, sampling from the target non-Gaussian distribution can be efficiently achieved by diffusing samples obtained from the pre-trained generative model. In this way, the number of required denoising steps is significantly reduced. In the meantime, the sample quality of ES-DDPM also improves substantially, outperforming both the vanilla DDPM and the adopted pre-trained generative model. On extensive experiments across CIFAR-10, CelebA, ImageNet, LSUN-Bedroom and LSUN-Cat, ES-DDPM obtains promising acceleration effect and performance improvement over representative baseline methods. Moreover, ES-DDPM also demonstrates several attractive properties, including being orthogonal to existing acceleration methods, as well as simultaneously enabling both global semantic and local pixel-level control in image generation.
Implication of Spin Constraints by the Event Horizon Telescope on Stellar Orbits in the Galactic Center

The center of the Milky Way hosts the closest supermassive black hole, Sgr A*. Decades of near-infrared observations of our Galactic Center have shown the presence of a small population of stars (the so-called S-star cluster) orbiting Sgr A*, which were recently reported to be arranged into two orthogonal disks. In this case, the timescale for the Lense–Thirring precession of S stars should be longer than their age, implying a low spin for Sgr A*. In contrast, the recent results by the Event Horizon Telescope favor a highly spinning Sgr A*, which seems to suggest that the S stars could not be arranged in disks. Alternatively, the spin of Sgr A* must be small, suggesting that the models for its observed image are incomplete.
Train Flat, Then Compress: Sharpness-Aware Minimization Learns More Compressible Models

Model compression by way of parameter pruning, quantization, or distillation has recently gained popularity as an approach for reducing the computational requirements of modern deep neural network models for NLP. Inspired by prior works suggesting a connection between simpler, more generalizable models and those that lie within wider loss basins, we hypothesize that optimizing for flat minima should lead to simpler parameterizations and thus more compressible models. We propose to combine sharpness-aware minimization (SAM) with various task-specific model compression methods, including iterative magnitude pruning (IMP), structured pruning with a distillation objective, and post-training dynamic quantization. Empirically, we show that optimizing for flatter minima consistently leads to greater compressibility of parameters compared to vanilla Adam when fine-tuning BERT models, with little to no loss in accuracy on the GLUE text classification and SQuAD question answering benchmarks. Moreover, SAM finds superior winning tickets during IMP that 1) are amenable to vanilla Adam optimization, and 2) transfer more effectively across tasks.
An improved broadband transition between microstrip and empty substrate integrated waveguide

In this letter, an improved broadband transition between microstrip and empty substrate integrated waveguide (ESIW) is proposed. The transition structure consists of a gradient microstrip line, a taper extended into ESIW, and ESIW section. Compared with the previous studies, the gradient microstrip line is introduced to expand the operating bandwidth and get a better performance, especially for the thinner dielectric substrate. The simulated and measured results have a good agreement. A back‐to‐back transition is designed, manufactured, and tested. For a single transition, the return loss and insertion loss are better than 13.5 and 0.75 dB from 6.6 to 16.45 GHz (85.5%), respectively. © 2016 Wiley Periodicals, Inc. Microwave Opt Technol Lett 58:2227–2231, 2016
The Contributions of Mu-Opioid Receptors on Glutamatergic and GABAergic Neurons to Analgesia Induced by Various Stress Intensities

Abstract The endogenous opioid system plays a crucial role in stress-induced analgesia. Mu-opioid receptors (MORs), one of the major opioid receptors, are expressed widely in subpopulations of cells throughout the CNS. However, the potential roles of MORs expressed in glutamatergic (MORGlut) and γ-aminobutyric acidergic (MORGABA) neurons in stress-induced analgesia remain unclear. By examining tail-flick latencies to noxious radiant heat of male mice, here we investigated the contributions of MORGABA and MORGlut to behavioral analgesia and activities of neurons projecting from periaqueductal gray (PAG) to rostral ventromedial medulla (RVM) induced by a range of time courses of forced swim exposure. The moderate but not transitory or prolonged swim exposure induced a MOR-dependent analgesia, although all of these three stresses enhanced β-endorphin release. Selective deletion of MORGABA but not MORGlut clearly attenuated analgesia and blocked the enhancement of activities of PAG-RVM neurons induced by moderate swim exposure. Under transitory swim exposure, in contrast, selective deletion of MORGlut elicited an analgesia behavior via strengthening the activities of PAG-RVM neurons. These results indicate that MOR-dependent endogenous opioid signaling participates in nociceptive modulation in a wide range, not limited to moderate, of stress intensities. Endogenous activation of MORGABA exerts analgesia, whereas MORGlut produces antianalgesia. More importantly, with an increase of stress intensities, the efficiencies of MORs on nociception shifts from balance between MORGlut and MORGABA to biasing toward MORGABA-mediated processes. Our results point to the cellular dynamic characteristics of MORs expressed in excitatory and inhibitory neurons in pain modulation under various stress intensities.
From dorsomobility to dorsostability: A study of lumbosacral joint range of motion in artiodactyls

This study is the first analysis of mobility in the lumbosacral joint of even‐toed ungulates covering the full range of body masses and running forms. In this study, we modified a previously developed osteometry‐based method to calculate the available range of motion (aROM) in the lumbosacral joint in artiodactyls. We quantified all three directions of intervertebral mobility: sagittal bending (SB), lateral bending (LB), and axial rotation (AR). This research covers extant artiodactyls from 10 families, 57 genera, and 78 species. The lumbosacral joint in artiodactyls is on average almost twice as mobile in SB as the average intralumbar joint (aROM 15.68° vs 8.22°). In all artiodactyls, the first sacral prezygapophyses are equipped with postfacet fossae determining the available range of lumbosacral hyperextension. SB aROM in the lumbosacral joint in artiodactyls varies almost sevenfold (from 4.53° to 31.19°) and is closely related to the body mass and running form. An allometric equation was developed for the first time, for the joint angular amplitude of motion, exemplified by the artiodactyl lumbosacral SB aROMs, as a power function of body mass, the power coefficient value being close to −0.15. High SB aROM at the lumbosacral joint is characteristic of artiodactyls with at least one of the following characteristics: high cumulative and average SB aROM in the lumbar region (Pearson r = 0.467–0.617), small body mass (r = −0.531), saltatorial or saltatorial‐cursorial running form (mean = 16.91–18.63°). The highest SB aROM in the lumbosacral joint is typical for small antelopes and Moschidae (mean = 20.24–20.27°). Among these artiodactyls SB aROMs in the lumbosacral joint are on par with various carnivores. Large and robust artiodactyls, adapted predominantly to mediportal and stilt (running on extremely tall limbs) running forms, have 2–3 times smaller SB aROMs in the lumbosacral joint. Adaptation to endurance galloping in open landscapes (cursorial running form) is accompanied by smaller lumbar and lumbosacral SB aROMs compared to that in saltatorial‐cursorial artiodactyls of the same body mass. The wide range of species studied makes it possible to significantly expand the knowledge of relations of the mobility of the lumbosacral joint in artiodactyls to body mass and running form.
Short-term memory conjunctive binding in Alzheimer's disease: A systematic review and meta-analysis.

OBJECTIVE Short-term memory (STM) binding tests assess the ability to temporarily hold conjunctions between surface features, such as objects and their colors (i.e., feature binding condition), relative to the ability to hold the individual features (i.e., single feature condition). Impairments in performance of these tests have been considered cognitive markers of Alzheimer's disease (AD). The objective of the present study was to conduct a meta-analysis of results from STM binding tests used in the assessment of samples mapped along the AD clinical continuum. METHOD We searched PubMed, Scopus, and Web of Science for articles that assessed patients with AD (from preclinical to dementia) using the STM binding tests and compared their results with those of controls. From each relevant article, we extracted the number of participants, the mean and standard deviations from single feature and of feature binding conditions. Results across studies were combined using standardized mean differences (effect sizes) to produce overall estimates of effect. RESULTS The feature binding condition of the STM binding showed large effects in all stages of AD. However, small sample sizes across studies, the presence of moderate to high heterogeneity and cross-sectional, case-controls designs decreased our confidence in the current evidence. CONCLUSIONS To be considered as a cognitive marker for AD, properly powered longitudinal designs and studies that clearly relate conjunctive memory tests with biomarkers (amyloid and tau) are still needed. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
A survey of metalworking fluid mist in manufacturing plants

Metalworking fluid mist levels in eight manufacturing plants were surveyed using a RAM-1 or a DataRAM, light scattering device. Most levels fell in the 1 to 2.50 mg/m 3 range, well within the present standards set by OSHA and ACGH. Variations in observed mist levels were not determined by metalworking fluid type but by variables such as ventilation, tempera-ture, humidity, fluid flow rate, and mist collection systems.
Computational Investigation of Performance and Emission Characteristics of Diesel Engine Running on Soybean Biodiesel with Varying Injection Timing

On account of the energy crisis and search for possible next generation alternative fuel, biodiesels have emerged as a popular choice for research in case of diesel engines. The characteristics of biodiesel are similar to diesel and thus they have been the subject of research for many years. Efforts are being made to increase the Brake Thermal efficiency and decrease the NOx emissions from diesel engines when running on biodiesels. For this, extensive experimentation is required which is cost ineffective and exhaustive. This work investigates the performance and emissions of a single cylinder diesel engine running on Soybean Biodiesel by varying the Injection Timing via the numerical approach using Diesel-RK software. Simulations were carried out for three different injection timings of 23°, 26°, and 29° bTDC for Soybean Methyl Ester (SME), at different load conditions in terms of brake power and constant speed of 1500 rpm and the results have been compared with those of baseline diesel fuel. The predicted results show better engine emission characteristics for the advancement of injection timing except NOx emissions while engine performance is better for retardation of timing except PM and Smoke emissions. The performance of diesel fuel is better than Soybean biodiesel, while SME exhibits low Smoke and PM emissions. Keywords— Biodiesel, SME, Injection Timing, Simulation, Diesel-RK software.
Uptake of hepatitis B antiviral treatment: A panel data analysis of 31 provinces in China (2013–2020)

China has made substantial efforts aimed to promote the uptake of antiviral treatment of hepatitis B (HB). It is unclear whether these policies achieved the desired impact. This study adopted medicines procurement data from 31 provinces to generate the first evidence about the number of standard antiviral treatment of HB overtime at both national and provincial levels in China.