Miracle-dz/UniFAR-FTUs huggingface dataset:
ClusterLLM: Large Language Models as a Guide for Text Clustering

We introduce ClusterLLM, a novel text clustering framework that leverages feedback from an instruction-tuned large language model, such as ChatGPT. Compared with traditional unsupervised methods that builds upon"small"embedders, ClusterLLM exhibits two intriguing advantages: (1) it enjoys the emergent capability of LLM even if its embeddings are inaccessible; and (2) it understands the user's preference on clustering through textual instruction and/or a few annotated data. First, we prompt ChatGPT for insights on clustering perspective by constructing hard triplet questions, where A, B and C are similar data points that belong to different clusters according to small embedder. We empirically show that this strategy is both effective for fine-tuning small embedder and cost-efficient to query ChatGPT. Second, we prompt ChatGPT for helps on clustering granularity by carefully designed pairwise questions, and tune the granularity from cluster hierarchies that is the most consistent with the ChatGPT answers. Extensive experiments on 14 datasets show that ClusterLLM consistently improves clustering quality, at an average cost of ~$0.6 per dataset.
"What if?" in Probabilistic Logic Programming

A ProbLog program is a logic program with facts that only hold with a specified probability. In this contribution, we extend this ProbLog language by the ability to answer “What if” queries. Intuitively, a ProbLog program defines a distribution by solving a system of equations in terms of mutually independent predefined Boolean random variables. In the theory of causality, Judea Pearl proposes a counterfactual reasoning for such systems of equations. Based on Pearl’s calculus, we provide a procedure for processing these counterfactual queries on ProbLog programs, together with a proof of correctness and a full implementation. Using the latter, we provide insights into the influence of different parameters on the scalability of inference. Finally, we also show that our approach is consistent with CP-logic, that is with the causal semantics for logic programs with annotated with disjunctions.
Generalized Bayesian Inference for Scientific Simulators via Amortized Cost Estimation

Simulation-based inference (SBI) enables amortized Bayesian inference for simulators with implicit likelihoods. But when we are primarily interested in the quality of predictive simulations, or when the model cannot exactly reproduce the observed data (i.e., is misspecified), targeting the Bayesian posterior may be overly restrictive. Generalized Bayesian Inference (GBI) aims to robustify inference for (misspecified) simulator models, replacing the likelihood-function with a cost function that evaluates the goodness of parameters relative to data. However, GBI methods generally require running multiple simulations to estimate the cost function at each parameter value during inference, making the approach computationally infeasible for even moderately complex simulators. Here, we propose amortized cost estimation (ACE) for GBI to address this challenge: We train a neural network to approximate the cost function, which we define as the expected distance between simulations produced by a parameter and observed data. The trained network can then be used with MCMC to infer GBI posteriors for any observation without running additional simulations. We show that, on several benchmark tasks, ACE accurately predicts cost and provides predictive simulations that are closer to synthetic observations than other SBI methods, especially for misspecified simulators. Finally, we apply ACE to infer parameters of the Hodgkin-Huxley model given real intracellular recordings from the Allen Cell Types Database. ACE identifies better data-matching parameters while being an order of magnitude more simulation-efficient than a standard SBI method. In summary, ACE combines the strengths of SBI methods and GBI to perform robust and simulation-amortized inference for scientific simulators.
Evaluating OpenAI's Whisper ASR for Punctuation Prediction and Topic Modeling of life histories of the Museum of the Person

Automatic speech recognition (ASR) systems play a key role in applications involving human-machine interactions. Despite their importance, ASR models for the Portuguese language proposed in the last decade have limitations in relation to the correct identification of punctuation marks in automatic transcriptions, which hinder the use of transcriptions by other systems, models, and even by humans. However, recently Whisper ASR was proposed by OpenAI, a general-purpose speech recognition model that has generated great expectations in dealing with such limitations. This chapter presents the first study on the performance of Whisper for punctuation prediction in the Portuguese language. We present an experimental evaluation considering both theoretical aspects involving pausing points (comma) and complete ideas (exclamation, question, and fullstop), as well as practical aspects involving transcript-based topic modeling - an application dependent on punctuation marks for promising performance. We analyzed experimental results from videos of Museum of the Person, a virtual museum that aims to tell and preserve people's life histories, thus discussing the pros and cons of Whisper in a real-world scenario. Although our experiments indicate that Whisper achieves state-of-the-art results, we conclude that some punctuation marks require improvements, such as exclamation, semicolon and colon.
Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator

The transformer model is known to be computationally demanding, and prohibitively costly for long sequences, as the self-attention module uses a quadratic time and space complexity with respect to sequence length. Many researchers have focused on designing new forms of self-attention or introducing new parameters to overcome this limitation, however a large portion of them prohibits the model to inherit weights from large pretrained models. In this work, the transformer's inefficiency has been taken care of from another perspective. We propose Fourier Transformer, a simple yet effective approach by progressively removing redundancies in hidden sequence using the ready-made Fast Fourier Transform (FFT) operator to perform Discrete Cosine Transformation (DCT). Fourier Transformer is able to significantly reduce computational costs while retain the ability to inherit from various large pretrained models. Experiments show that our model achieves state-of-the-art performances among all transformer-based models on the long-range modeling benchmark LRA with significant improvement in both speed and space. For generative seq-to-seq tasks including CNN/DailyMail and ELI5, by inheriting the BART weights our model outperforms the standard BART and other efficient models. \footnote{Our code is publicly available at \url{https://github.com/LUMIA-Group/FourierTransformer}}
Unsupervised Discovery of Continuous Skills on a Sphere

Recently, methods for learning diverse skills to generate various behaviors without external rewards have been actively studied as a form of unsupervised reinforcement learning. However, most of the existing methods learn a finite number of discrete skills, and thus the variety of behaviors that can be exhibited with the learned skills is limited. In this paper, we propose a novel method for learning potentially an infinite number of different skills, which is named discovery of continuous skills on a sphere (DISCS). In DISCS, skills are learned by maximizing mutual information between skills and states, and each skill corresponds to a continuous value on a sphere. Because the representations of skills in DISCS are continuous, infinitely diverse skills could be learned. We examine existing methods and DISCS in the MuJoCo Ant robot control environments and show that DISCS can learn much more diverse skills than the other methods.
DeepCollide: Scalable Data-Driven High DoF Configuration Space Modeling using Implicit Neural Representations

Collision detection is essential to virtually all robotics applications. However, traditional geometric collision detection methods generally require pre-existing workspace geometry representations; thus, they are unable to infer the collision detection function from sampled data when geometric information is unavailable. Learning-based approaches can overcome this limitation. Following this line of research, we present DeepCollide, an implicit neural representation method for approximating the collision detection function from sampled collision data. As shown by our theoretical analysis and empirical evidence, DeepCollide presents clear benefits over the state-of-the-art, as it relates to time cost scalability with respect to training data and DoF, as well as the ability to accurately express complex workspace geometries. We publicly release our code.
ViTMatte: Boosting Image Matting with Pretrained Plain Vision Transformers

Recently, plain vision Transformers (ViTs) have shown impressive performance on various computer vision tasks, thanks to their strong modeling capacity and large-scale pretraining. However, they have not yet conquered the problem of image matting. We hypothesize that image matting could also be boosted by ViTs and present a new efficient and robust ViT-based matting system, named ViTMatte. Our method utilizes (i) a hybrid attention mechanism combined with a convolution neck to help ViTs achieve an excellent performance-computation trade-off in matting tasks. (ii) Additionally, we introduce the detail capture module, which just consists of simple lightweight convolutions to complement the detailed information required by matting. To the best of our knowledge, ViTMatte is the first work to unleash the potential of ViT on image matting with concise adaptation. It inherits many superior properties from ViT to matting, including various pretraining strategies, concise architecture design, and flexible inference strategies. We evaluate ViTMatte on Composition-1k and Distinctions-646, the most commonly used benchmark for image matting, our method achieves state-of-the-art performance and outperforms prior matting works by a large margin.
Constrained Ergodic optimization for generic continuous functions

One of the fundamental results of ergodic optimization asserts that for any dynamical system on a compact metric space with the specification property and for a generic continuous function $f$ every invariant probability measure that maximizes the space average of $f$ must have zero entropy. We establish the analogical result in the context of constraint ergodic optimization, which is introduced by Garibaldi and Lopes (2007).
The Use of Mid-Pregnancy Cervical Length to Predict Preterm Birth in Brazilian Asymptomatic Twin Gestations

Abstract Objective To describe a reference curve for cervical length (CL) in mid-trimester twin gestations using transvaginal ultrasound (TVU) and to investigate whether short CL increases spontaneous preterm birth (sPTB) in asymptomatic twin pregnancies. Methods This was a prospective cohort study performed at 17 outpatient antenatal facilities of Brazil with women at 18 0/7 to 22 6/7 weeks of gestation who participated in a randomized clinical trial screening phase (P5 trial) between July 2015 and March 2019. TVU was performed to provide CL measurement in all screened women. Almost all women with CL ≤ 30 mm received vaginal progesterone 200mg/day and they were also randomized to receive cervical pessary or not. We considered data from the CL distribution among asymptomatic twin pregnancies and analyzed CL and its association with PTB generating receiver operating characteristics (ROC) curves and Kaplan-Meier curves. Results A total of 253 pregnant women with twins were included in the distribution curve. The mean CL was 33.7 mm and median was 35.5mm. The 10th percentile was 17.8mm. We identified a PTB rate of 73.9% (187/253) with 33.6% of sPTB < 37 (85/253) and 15% (38/253) of sPTB < 34 weeks. The best cutoff point to predict sPTB < 37 was 24.15 mm. However, the ROC curve showed a poor performance (0.64). The Kaplan-Meier survival curves identified that only CL values ≤ 20mm were associated to sPTB < 34 weeks. Conclusion A cutoff point of CL ≤ 20 mm can be interesting point to identify short cervix in Brazilian twin pregnancies. However, in Brazilian asymptomatic twin pregnancies, CL does not show a good performance to predict PTB.
Concentration Inequalities: A Nonasymptotic Theory of Independence

Concentration inequalities for functions of independent random variables is an area of probability theory that has witnessed a great revolution in the last few decades, and has applications in a wide variety of areas such as machine learning, statistics, discrete mathematics, and high-dimensional geometry. Roughly speaking, if a function of many independent random variables does not depend too much on any of the variables then it is concentrated in the sense that with high probability, it is close to its expected value. This book offers a host of inequalities to illustrate this rich theory in an accessible way by covering the key developments and applications in the field. The authors describe the interplay between the probabilistic structure (independence) and a variety of tools ranging from functional inequalities to transportation arguments to information theory. Applications to the study of empirical processes, random projections, random matrix theory, and threshold phenomena are also presented. A self-contained introduction to concentration inequalities, it includes a survey of concentration of sums of independent random variables, variance bounds, the entropy method, and the transportation method. Deep connections with isoperimetric problems are revealed whilst special attention is paid to applications to the supremum of empirical processes. Written by leading experts in the field and containing extensive exercise sections this book will be an invaluable resource for researchers and graduate students in mathematics, theoretical computer science, and engineering.
Integral Probability Metrics and Their Generating Classes of Functions

We consider probability metrics of the following type: for a class of functions and probability measures P, Q we define A unified study of such integral probability metrics is given. We characterize the maximal class of functions that generates such a metric. Further, we show how some interesting properties of these probability metrics arise directly from conditions on the generating class of functions. The results are illustrated by several examples, including the Kolmogorov metric, the Dudley metric and the stop-loss metric.
Traditional egg-harvesting policies for the endemic Moluccan Scrubfowl Eulipoa wallacei on Haruku Island, Indonesia, and its conservation implications

Summary Traditional policies on using wild species are critical in developing scenarios for community engagement in conserving wild species populations. As such, this study examines the traditional practice of harvesting eggs from Moluccan Scrubfowl Eulipoa wallacei on Haruku Island, Indonesia, and its implications for bird population conservation. Using a closed-ended questionnaire, we surveyed 200 respondents in the productive age group (aged 15–65 years). It was found that most of the community supported adopting the traditional policies due to the high demand for eggs, which required daily harvesting. According to the community, egg harvesting has no adverse effect on bird populations, but hunting and habitat destruction does reduce the number of eggs available for harvesting. The community concluded that traditional conservation policies prioritise the conservation of adult birds and their habitats, not bird eggs. Particular attention should be paid to egg conservation efforts by establishing a quota for egg harvesting and limiting egg-harvesting time. Conservation efforts must also take into account the growing community awareness of the importance of conserving Moluccan Scrubfowl eggs.
Mechanistic explanation of neuroplasticity using equivalent circuits

This paper introduces a comprehensive mechanistic model of a neuron with plasticity that explains the biophysical correlates of memory for a single neuron. This means clarifying how information input as time-varying signals is processed, stored, and subsequently recalled. Moreover, the model addresses two additional, long-standing, specific biological problems: the integration of Hebbian and homeostatic plasticity, and the identification of a concise learning rule for synapses. In this study, a biologically accurate Hodgkin-Huxley-style electric-circuit equivalent is derived through a one-to-one mapping from the known properties of ion channels. The dynamics of the synaptic cleft, which is often overlooked, is found to be essential in this process. Analysis of the model reveals a simple and succinct learning rule, indicating that the neuron functions as an internal-feedback adaptive filter, which is commonly used in signal processing. Simulation results confirm the circuit’s functionality, stability, and convergence, demonstrating that even a single neuron without external feedback can function as a potent signal processor. The article is interdisciplinary and spans a broad range of subjects within the realm of biophysics, including neurobiology, electronics, and signal processing. Significance statement Mechanistic neuron models with plasticity are crucial for understanding the complexities of the brain and the processes behind learning and memory. These models provide a way to study how individual neurons and synapses in the brain change over time in response to stimuli, allowing for a more nuanced understanding of neuronal circuits and assemblies. Plasticity is a key aspect of these models, as it represents the ability of the brain to modify its connections and functions in response to experiences. By incorporating plasticity into these models, researchers can explore how changes at the synaptic level contribute to higher-level changes in behavior and cognition. Thus, these models are essential for advancing our understanding of the brain and its functions. PhySH 2023 terms Neuroplasticity, Learning, Memory, Synapses. MeSH 2023 terms Neuronal Plasticity [G11.561. 638], Association learning [F02.463. 425.069. 296], Memory [F02.463. 425.540], Synaptic transmission [G02.111. 820.850]
Learning Equivariant Object Recognition and its Reverse Application to Imagery

To investigate the relationship of perception and imagery, we model the visual ventral stream with an encoder and decoder part with the help of capsule networks. The proposed network consists of V1 and V2 from CorNet-Z, as well as the Capsule Network architecture with the routing by agreement algorithm for V4 and IT. The decoder reverses this architecture to model the feedback activation patterns of the visual ventral stream. The model was trained using EMNIST (letters H, S, C, T). Resulting classification performance was high with good generalization performance to different sizes, positions, and rotations. Contextual information was used for occluded stimuli in the feedback path for reconstructions resulting in high classification performance. Additionally, a pre-trained network was used to reconstruct remapped fMRI activation patterns from higher visual areas. Reconstructions of single-trial imagery data showed significant correlations to physical letter stimuli. The fMRI activation patterns of V1 and V2 and their reconstructions with population receptive field mapping and an autoencoder were related to activation patterns of the network to test biological plausibility. Representational Similarity Analysis and spatial correlations indicated an overlap of information content between the capsule network and the fMRI activations. Due to the capsule networks’ high generalization performance and the implemented feedback connections, the proposed network is a promising approach to improve current modelling efforts of perception and imagery. Further research is needed to compare the presented network to established networks that model the visual ventral stream.
Uncertain, intermittent access to reward promotes increased reward pursuit

Self-administration procedures have been developed to model the intermittency of cocaine use in humans. These procedures involve intermittent, predictable access to cocaine during daily self-administration sessions. However, human drug use often involves intermittent and unpredictable patterns of drug access. Here, we introduce a new procedure to study the effects of unpredictable, intermittent access (UIntA) to a reinforcer, and we compare this procedure to two existing ones that provide predictable reinforcer availability; continuous (ContA) or intermittent (IntA) access. Three groups of rats self-administered water or a 5% sucrose solution in daily hour-long sessions. UIntA rats had alternating periods of reinforcer ON and OFF of unpredictable duration (1, 5 or 9 min/period). During reinforcer ON periods, reinforcer quantities were also unpredictable (0, 0.1 or 0.2 ml of solution) and were available under a variable ratio 3 schedule of reinforcement (1-6 responses). Both IntA and ContA rats had access to a fixed volume of water or sucrose (0.1 ml), under a fixed ratio 3 schedule of reinforcement. IntA rats had alternating and predictable 5-min reinforcer ON and OFF periods, while ContA rats had 60 minutes of reinforcer access during each session. Following 14 daily self-administration sessions, we found that UIntA rats had the highest rates of responding for water or sucrose reward under progressive ratio and extinction conditions, and the highest levels of cue-induced reinstatement of sucrose seeking. Thus, uncertain, intermittent access to reward promotes increased reward-seeking and -taking behaviours. This has implications for modeling addiction and other disorders of increased reward seeking.