Miracle-dz/UniFAR-FTUs huggingface dataset:
Efficient Benchmarking (of Language Models)

The increasing versatility of language models LMs has given rise to a new class of benchmarks that comprehensively assess a broad range of capabilities. Such benchmarks are associated with massive computational costs reaching thousands of GPU hours per model. However the efficiency aspect of these evaluation efforts had raised little discussion in the literature. In this work we present the problem of Efficient Benchmarking namely intelligently reducing the computation costs of LM evaluation without compromising reliability. Using the HELM benchmark as a test case we investigate how different benchmark design choices affect the computation-reliability tradeoff. We propose to evaluate the reliability of such decisions by using a new measure Decision Impact on Reliability DIoR for short. We find for example that the current leader on HELM may change by merely removing a low-ranked model from the benchmark and observe that a handful of examples suffice to obtain the correct benchmark ranking. Conversely a slightly different choice of HELM scenarios varies ranking widely. Based on our findings we outline a set of concrete recommendations for more efficient benchmark design and utilization practices leading to dramatic cost savings with minimal loss of benchmark reliability often reducing computation by x100 or more.
State Merging with Quantifiers in Symbolic Execution

We address the problem of constraint encoding explosion which hinders the applicability of state merging in symbolic execution. Specifically, our goal is to reduce the number of disjunctions and if-then-else expressions introduced during state merging. The main idea is to dynamically partition the symbolic states into merging groups according to a similar uniform structure detected in their path constraints, which allows to efficiently encode the merged path constraint and memory using quantifiers. To address the added complexity of solving quantified constraints, we propose a specialized solving procedure that reduces the solving time in many cases. Our evaluation shows that our approach can lead to significant performance gains.
VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detection

The recent contrastive language-image pre-training (CLIP) model has shown great success in a wide range of image-level tasks, revealing remarkable ability for learning powerful visual representations with rich semantics. An open and worthwhile problem is efficiently adapting such a strong model to the video domain and designing a robust video anomaly detector. In this work, we propose VadCLIP, a new paradigm for weakly supervised video anomaly detection (WSVAD) by leveraging the frozen CLIP model directly without any pre-training and fine-tuning process. Unlike current works that directly feed extracted features into the weakly supervised classifier for frame-level binary classification, VadCLIP makes full use of fine-grained associations between vision and language on the strength of CLIP and involves dual branch. One branch simply utilizes visual features for coarse-grained binary classification, while the other fully leverages the fine-grained language-image alignment. With the benefit of dual branch, VadCLIP achieves both coarse-grained and fine-grained video anomaly detection by transferring pre-trained knowledge from CLIP to WSVAD task. We conduct extensive experiments on two commonly-used benchmarks, demonstrating that VadCLIP achieves the best performance on both coarse-grained and fine-grained WSVAD, surpassing the state-of-the-art methods by a large margin. Specifically, VadCLIP achieves 84.51% AP and 88.02% AUC on XD-Violence and UCF-Crime, respectively. Code and features are released at https://github.com/nwpu-zxr/VadCLIP.
Exploring the impact of imaging cadence on inferring CME kinematics

The kinematics of coronal mass ejections (CMEs) are crucial for understanding their initiation mechanism and predicting their impact on Earth and other planets. With most of the acceleration and deceleration occurring below 4 R⊙, capturing this phase is vital to better understand their initiation mechanism. Furthermore, the kinematics of CMEs in the inner corona (< 3 R⊙) are closely related to their propagation in the outer corona and their eventual impact on Earth. Since the kinematics of CMEs are mainly probed using coronagraph data, it is crucial to investigate the impact of imaging cadence on the precision of data analysis and the conclusions drawn from it and also for determining the flexibility of designing observational campaigns with upcoming coronagraphs. This study investigates the impact of imaging cadence on the kinematics of ten CMEs observed by the K-Coronagraph of the Mauna Loa Solar Observatory. We manually track the CMEs using high cadence (15 s) white-light observations of K-Cor and vary the cadence as 30 s, 1, 2, and 5 min to study the impact of cadence on the kinematics. We also employed the bootstrapping method to estimate the confidence interval of the fitting parameters. Our results indicate that the average velocity of the CMEs does not have a high dependence on the imaging cadence, while the average acceleration shows significant dependence on the same, with the confidence interval showing significant shifts for the average acceleration for different cadences. The impact of degraded cadence is also seen in the estimation of the time of onset of acceleration. We further find that it is difficult to find an optimum cadence to study all CMEs, as it is also influenced by the pixel resolution of the instrument and the speed of the CME. However, except for very slow CMEs (speeds less than 300 km−1), our results indicate a cadence of 1 min to be reasonable for the study of their kinematics. The results of this work will be important in the planning of observational campaigns for the existing and upcoming missions that will observe the inner corona.
Adaptive Encoding Strategies for Erasing-Based Lossless Floating-Point Compression

Lossless floating-point time series compression is crucial for a wide range of critical scenarios. Nevertheless, it is a big challenge to compress time series losslessly due to the complex underlying layouts of floating-point values. The state-of-the-art erasing-based compression algorithm Elf demonstrates a rather impressive performance. We give an in-depth exploration of the encoding strategies of Elf, and find that there is still much room for improvement. In this paper, we propose Elf*, which employs a set of optimizations for leading zeros, center bits and sharing condition. Specifically, we develop a dynamic programming algorithm with a set of pruning strategies to compute the adaptive approximation rules efficiently. We theoretically prove that the adaptive approximation rules are globally optimal. We further extend Elf* to Streaming Elf*, i.e., SElf*, which achieves almost the same compression ratio as Elf*, while enjoying even higher efficiency in streaming scenarios. We compare Elf* and SElf* with 8 competitors using 22 datasets. The results demonstrate that SElf* achieves 9.2% relative compression ratio improvement over the best streaming competitor while maintaining similar efficiency, and that Elf* ranks among the most competitive batch compressors. All source codes are publicly released.
Characterizing normal perinatal development of the human brain structural connectivity

Early brain development is characterized by the formation of a highly organized structural connectome. The interconnected nature of this connectome underlies the brain’s cognitive abilities and influences its response to diseases and environmental factors. Hence, quantitative assessment of structural connectivity in the perinatal stage is useful for studying normal and abnormal neurodevelopment. However, estimation of the connectome from diffusion MRI data involves complex computations. For the perinatal period, these computations are further challenged by the rapid brain development and imaging difficulties. Combined with high inter-subject variability, these factors make it difficult to chart the normal development of the structural connectome. As a result, there is a lack of reliable normative baselines of structural connectivity metrics at this critical stage in brain development. In this study, we developed a computational framework, based on spatio-temporal averaging, for determining such baselines. We used this framework to analyze the structural connectivity between 33 and 44 postmenstrual weeks using data from 166 subjects. Our results unveiled clear and strong trends in the development of structural connectivity in perinatal stage. Connection weighting based on fractional anisotropy and neurite density produced the most consistent results. We observed increases in global and local efficiency, a decrease in characteristic path length, and widespread strengthening of the connections within and across brain lobes and hemispheres. We also observed asymmetry patterns that were consistent between different connection weighting approaches. The new computational method and results are useful for assessing normal and abnormal development of the structural connectome early in life.
Rethinking Data Perturbation and Model Stabilization for Semi-supervised Medical Image Segmentation

Studies on semi-supervised medical image segmentation (SSMIS) have seen fast progress recently. Due to the limited labelled data, SSMIS methods mainly focus on effectively leveraging unlabeled data to enhance the segmentation performance. However, despite their promising performance, current state-of-the-art methods often prioritize integrating complex techniques and loss terms rather than addressing the core challenges of semi-supervised scenarios directly. We argue that the key to SSMIS lies in generating substantial and appropriate prediction disagreement on unlabeled data. To this end, we emphasize the crutiality of data perturbation and model stabilization in semi-supervised segmentation, and propose a simple yet effective approach to boost SSMIS performance significantly, dubbed DPMS. Specifically, we first revisit SSMIS from three distinct perspectives: the data, the model, and the loss, and conduct a comprehensive study of corresponding strategies to examine their effectiveness. Based on these examinations, we then propose DPMS, which adopts a plain teacher-student framework with a standard supervised loss and unsupervised consistency loss. To produce appropriate prediction disagreements, DPMS perturbs the unlabeled data via strong augmentations to enlarge prediction disagreements considerably. On the other hand, using EMA teacher when strong augmentation is applied does not necessarily improve performance. DPMS further utilizes a forwarding-twice and momentum updating strategies for normalization statistics to stabilize the training on unlabeled data effectively. Despite its simplicity, DPMS can obtain new state-of-the-art performance on the public 2D ACDC and 3D LA datasets across various semi-supervised settings, e.g. obtaining a remarkable 22.62% improvement against previous SOTA on ACDC with 5% labels.
Electromagnetic Casimir–Polder Interaction for a Conducting Cone

Using the formulation of the electromagnetic Green’s function of a perfectly conducting cone in terms of analytically continued angular momentum, we compute the Casimir–Polder interaction energy of a cone with a polarizable particle. We introduce this formalism by first reviewing the analogous approach for a perfectly conducting wedge, and then demonstrate the calculation through numerical evaluation of the resulting integrals.
Example-Based Framework for Perceptually Guided Audio Texture Generation

Generative models for synthesizing audio textures explicitly encode controllability by conditioning the model with labelled data. While datasets for audio textures can be easily recorded in-the-wild, semantically labeling them is expensive, time-consuming, and prone to errors due to human annotator subjectivity. Thus, to control generation, there is a need to automatically infer user-defined perceptual factors of variation in the latent space of a generative model while modelling unlabeled textures. In this paper, we propose an example-based framework to determine vectors to guide texture generation based on user-defined semantic attributes. By synthesizing a few synthetic examples to indicate the presence or absence of a semantic attribute, we can infer the guidance vectors in the latent space of a generative model to control that attribute during generation. Our results show that our method is capable of finding perceptually relevant and deterministic guidance vectors for controllable generation for both discrete as well as continuous textures. Furthermore, we demonstrate the application of this method to other tasks such as selective semantic attribute transfer.
SEA: Shareable and Explainable Attribution for Query-based Black-box Attacks

Machine Learning (ML) systems are vulnerable to adversarial examples, particularly those from query-based black-box attacks. Despite various efforts to detect and prevent such attacks, there is a need for a more comprehensive approach to logging, analyzing, and sharing evidence of attacks. While classic security benefits from well-established forensics and intelligence sharing, Machine Learning is yet to find a way to profile its attackers and share information about them. In response, this paper introduces SEA, a novel ML security system to characterize black-box attacks on ML systems for forensic purposes and to facilitate human-explainable intelligence sharing. SEA leverages the Hidden Markov Models framework to attribute the observed query sequence to known attacks. It thus understands the attack's progression rather than just focusing on the final adversarial examples. Our evaluations reveal that SEA is effective at attack attribution, even on their second occurrence, and is robust to adaptive strategies designed to evade forensics analysis. Interestingly, SEA's explanations of the attack behavior allow us even to fingerprint specific minor implementation bugs in attack libraries. For example, we discover that the SignOPT and Square attacks implementation in ART v1.14 sends over 50% specific zero difference queries. We thoroughly evaluate SEA on a variety of settings and demonstrate that it can recognize the same attack's second occurrence with 90+% Top-1 and 95+% Top-3 accuracy.
A Survey for Federated Learning Evaluations: Goals and Measures

Evaluation is a systematic approach to assessing how well a system achieves its intended purpose. Federated learning (FL) is a novel paradigm for privacy-preserving machine learning that allows multiple parties to collaboratively train models without sharing sensitive data. However, evaluating FL is challenging due to its interdisciplinary nature and diverse goals, such as utility, efficiency, and security. In this survey, we first review the major evaluation goals adopted in the existing studies and then explore the evaluation metrics used for each goal. We also introduce FedEval, an open-source platform that provides a standardized and comprehensive evaluation framework for FL algorithms in terms of their utility, efficiency, and security. Finally, we discuss several challenges and future research directions for FL evaluation.
Constrained Stein Variational Trajectory Optimization

We present Constrained Stein Variational Trajectory Optimization (CSVTO), an algorithm for performing trajectory optimization with constraints on a set of trajectories in parallel. We frame constrained trajectory optimization as a novel form of constrained functional minimization over trajectory distributions, which avoids treating the constraints as a penalty in the objective and allows us to generate diverse sets of constraint-satisfying trajectories. Our method uses Stein Variational Gradient Descent (SVGD) to find a set of particles that approximates a distribution over low-cost trajectories while obeying constraints. CSVTO is applicable to problems with arbitrary equality and inequality constraints and includes a novel particle resampling step to escape local minima. By explicitly generating diverse sets of trajectories, CSVTO is better able to avoid poor local minima and is more robust to initialization. We demonstrate that CSVTO outperforms baselines in challenging highly-constrained tasks, such as a 7DoF wrench manipulation task, where CSVTO succeeds in 20/20 trials vs 13/20 for the closest baseline. Our results demonstrate that generating diverse constraint-satisfying trajectories improves robustness to disturbances and initialization over baselines.
Improving Knowledge Extraction from LLMs for Task Learning through Agent Analysis

Large language models (LLMs) offer significant promise as a knowledge source for task learning. Prompt engineering has been shown to be effective for eliciting knowledge from an LLM, but alone it is insufficient for acquiring relevant, situationally grounded knowledge for an embodied agent learning novel tasks. We describe a cognitive-agent approach, STARS, that extends and complements prompt engineering, mitigating its limitations and thus enabling an agent to acquire new task knowledge matched to its native language capabilities, embodiment, environment, and user preferences. The STARS approach is to increase the response space of LLMs and deploy general strategies, embedded within the autonomous agent, to evaluate, repair, and select among candidate responses produced by the LLM. We describe the approach and experiments that show how an agent, by retrieving and evaluating a breadth of responses from the LLM, can achieve 77-94% task completion in one-shot learning without user oversight. The approach achieves 100% task completion when human oversight (such as an indication of preference) is provided. Further, the type of oversight largely shifts from explicit, natural language instruction to simple confirmation/discomfirmation of high-quality responses that have been vetted by the agent before presentation to a user.
P RUNING D EEP N EURAL N ETWORKS FROM A S PAR - SITY P ERSPECTIVE

In recent years, deep network pruning has attracted significant attention in order to enable the rapid deployment of AI into small devices with computation and memory constraints. Pruning is often achieved by dropping redundant weights, neurons, or layers of a deep network while attempting to retain a comparable test performance. Many deep pruning algorithms have been proposed with impressive empirical success. However, existing approaches lack a quantifiable measure to estimate the compressibility of a sub-network during each pruning iteration and thus may underprune or over-prune the model. In this work, we propose PQ Index (PQI) to measure the potential compressibility of deep neural networks and use this to develop a Sparsity-informed Adaptive Pruning (SAP) algorithm. Our extensive experiments corroborate the hypothesis that for a generic pruning procedure, PQI decreases first when a large model is being effectively regularized and then increases when its compressibility reaches a limit that appears to correspond to the beginning of underfitting. Subsequently, PQI decreases again when the model collapse and significant deterioration in the performance of the model start to occur. Additionally, our experiments demonstrate that the proposed adaptive pruning algorithm with proper choice of hyper-parameters is superior to the iterative pruning algorithms such as the lottery ticket-based pruning methods, in terms of both compression efficiency and robustness. Our code is available here.
Targeting sulfation-dependent mechanoreciprocity between matrix and osteoblasts to mitigate bone loss

Sulfation is a widespread modification of biomolecules that has been incompletely explored to date. Through cross-phenotype meta-analysis of bone mineral density in up to 426,824 genotyped human participants along with phenotypic characterization of multiple mutant mouse lines, we identified a causative role for sulfate transporter solute carrier family 26 member A2 (SLC26A2) deficiency in osteoporosis. Ablation of SLC26A2 in osteoblasts caused severe bone loss and accumulation of immature bone cells and elicited peculiar pericellular matrix (PCM) production characterized by undersulfation coupled with decreased stiffness. These altered chemophysical properties of the PCM disrupted the formation of focal adhesions in osteoblasts. Bulk RNA sequencing and functional assays revealed that the mechanoreciprocal inhibition of focal adhesion kinase (FAK) and Yes1-associated transcriptional regulator (YAP)/WW domain containing transcription regulator 1 (TAZ) signaling impinged osteoblast maturation upon SLC26A2 deficiency. Moreover, pharmacological abrogation of the Hippo kinases and forced wheel-running ameliorated SLC26A2-deficient osteoporosis by promoting YAP/TAZ activity. Analysis of mouse single-cell RNA sequencing data suggested coordination among sulfate metabolism, focal adhesion, and YAP/TAZ activity during osteoblast-to-osteocyte transition. In addition to the SLC26A2-deficient setting, altered FAK and YAP/TAZ signaling was also observed in bone cells of ovariectomized mice and patients with osteoporosis, and pharmacological enforcing of YAP/TAZ activity ameliorated bone loss in ovariectomized mice. Collectively, these data unveil a role for sulfation in the developmental mechanoreciprocity between matrix and osteoblasts, which could be leveraged to prevent bone loss. Description Sulfation wires developmental mechanoreciprocity between matrix and osteoblasts to promote bone anabolism. Editor’s summary Bone formation relies on osteoblasts’ ability to respond to mechanical forces transduced by focal adhesion complexes. Zheng et al. report that loss of function of the sulfate transporter SLC26A2 impedes bone anabolism by destabilizing focal adhesion–mediated interactions between matrix and osteoblasts, impeding osteoblast differentiation and function. In vivo experiments showed that SLC26A2 loss in osteoblasts resulted in impaired pericellular matrix sulfation, leading to altered matrix stiffness and decreased focal adhesion formation. Pharmacological augmentation of YAP/TAZ signaling abrogated bone loss in SLC26A2-deficient as well as ovariectomized mice. Genetic SLC26A2 deficiency associated with decreased bone mineral density in individuals with osteoporosis, suggesting potential for translation to humans. —Catherine Charneski
Management of Gastric Varices: GI Perspective

Abstract There is a paucity of literature on the management of gastric varices and current guidelines rely heavily on expert opinion. Optimal treatment of gastric varices requires a multidisciplinary team including gastrointestinal endoscopist, hepatologist, and interventional radiologists. Initial management relies on endoscopic diagnosis followed by therapeutic modalities based on the experience of the endoscopist and the institution. Definitive management may be endoscopic, endovascular, or a combination of the two disciplines. In this review, we discuss endoscopic management of gastric varices with direct endoscopic glue injection, endoscopic ultrasound-guided management, limitations, and adverse events. We conclude by describing a case of gastric variceal bleeding managed with endoscopy when interventional radiographic techniques were not feasible.