Miracle-dz/UniFAR-FTUs huggingface dataset:
Detecting and Exorcising Statistical Demons from Language Models with Anti-Models of Negative Data

It's been said that "Language Models are Unsupervised Multitask Learners. " Indeed, self-supervised language models trained on "positive" examples of English text generalize in desirable ways to many natural language tasks. But if such models can stray so far from an initial self-supervision objective, a wayward model might generalize in undesirable ways too, say to nonsensical "negative" examples of unnatural language. A key question in this work is: do language models trained on (positive) training data also generalize to (negative) test data? We use this question as a contrivance to assess the extent to which language models learn undesirable properties of text, such as n-grams, that might interfere with the learning of more desirable properties of text, such as syntax. We find that within a model family, as the number of parameters, training epochs, and data set size increase, so does a model's ability to generalize to negative n-gram data, indicating standard self-supervision generalizes too far. We propose a form of inductive bias that attenuates such undesirable signals with negative data distributions automatically learned from positive data. We apply the method to remove n-gram signals from LSTMs and find that doing so causes them to favor syntactic signals, as demonstrated by large error reductions (up to 46% on the hardest cases) on a syntactic subject-verb agreement task.
Optimization-Based Visual-Inertial SLAM Tightly Coupled with Raw GNSS Measurements

Unlike loose coupling approaches and the EKF-based approaches in the literature, we propose an optimization-based visual-inertial SLAM tightly coupled with raw Global Navigation Satellite System (GNSS) measurements, a first attempt of this kind in the literature to our knowledge. More specifically, reprojection error, IMU pre-integration error and raw GNSS measurement error are jointly minimized within a sliding window, in which the asynchronism between images and raw GNSS measurements is accounted for. In addition, issues such as marginalization, noisy measurements removal, as well as tackling vulnerable situations are also addressed. Experimental results on public dataset in complex urban scenes show that our proposed approach outperforms state-of-the-art visual-inertial SLAM, GNSS single point positioning, as well as a loose coupling approach, including scenes mainly containing low-rise buildings and those containing urban canyons.
Preliminary Target Selection for the DESI Luminous Red Galaxy (LRG) Sample

The DESI survey will observe more than 8 million candidate luminous red galaxies (LRGs) in the redshift range 0.3 < z < 1.0. Here we present a preliminary version of the DESI LRG target section developed using Legacy Surveys Data Release 8 g, r, z and W1 photometry. This selection yields a sample with a uniform surface density of ∼600 deg−2 and very low predicted stellar contamination and redshift failure rates. During DESI Survey Validation, updated versions of this selection will be tested and optimized.
The phase transition for planar Gaussian percolation models without FKG

We develop techniques to study the phase transition for planar Gaussian percolation models that are not (necessarily) positively correlated. These models lack the property of positive associations (also known as the `FKG inequality'), and hence many classical arguments in percolation theory do not apply. More precisely, we consider a smooth stationary centred planar Gaussian field $f$ and, given a level $\ell \in \mathbb{R}$, we study the connectivity properties of the excursion set $\{f \geq -\ell\}$. We prove the existence of a phase transition at the critical level $\ell_{crit}=0$ under only symmetry and (very mild) correlation decay assumptions, which are satisfied by the random plane wave for instance. As a consequence, all non-zero level lines are bounded almost surely, although our result does not settle the boundedness of zero level lines (`no percolation at criticality'). To show our main result: (i) we prove a general sharp threshold criterion, inspired by works of Chatterjee, that states that `sharp thresholds are equivalent to the delocalisation of the threshold location'; (ii) we prove threshold delocalisation for crossing events at large scales -- at this step we obtain a sharp threshold result but without being able to locate the threshold -- and (iii) to identify the threshold, we adapt Tassion's RSW theory replacing the FKG inequality by a sprinkling procedure. Although some arguments are specific to the Gaussian setting, many steps are very general and we hope that our techniques may be adapted to analyse other models without FKG.
An Industry Evaluation of Embedding-based Entity Alignment

Embedding-based entity alignment has been widely investigated in recent years, but most proposed methods still rely on an ideal supervised learning setting with a large number of unbiased seed mappings for training and validation, which significantly limits their usage. In this study, we evaluate those state-of-the-art methods in an industrial context, where the impact of seed mappings with different sizes and different biases is explored. Besides the popular benchmarks from DBpedia and Wikidata, we contribute and evaluate a new industrial benchmark that is extracted from two heterogeneous knowledge graphs (KGs) under deployment for medical applications. The experimental results enable the analysis of the advantages and disadvantages of these alignment methods and the further discussion of suitable strategies for their industrial deployment.
Meta-Learning Guarantees for Online Receding Horizon Control

In this paper we provide provable regret guarantees for an online meta-learning receding horizon control algorithm in an iterative control setting, where in each iteration the system to be controlled is a linear deterministic system that is different and unknown, the cost for the controller in an iteration is a general additive cost function and the control input is required to be constrained, which if violated incurs an additional cost. We prove (i) that the algorithm achieves a regret for the controller cost and constraint violation that are $O(T^{3/4})$ for an episode of duration $T$ with respect to the best policy that satisfies the control input control constraints and (ii) that the average of the regret for the controller cost and constraint violation with respect to the same policy vary as $O((1+1/\sqrt{N})T^{3/4})$ with the number of iterations $N$, showing that the worst regret for the learning within an iteration continuously improves with experience of more iterations.
Precise Statistical Analysis of Classification Accuracies for Adversarial Training

Despite the wide empirical success of modern machine learning algorithms and models in a multitude of applications, they are known to be highly susceptible to seemingly small indiscernible perturbations to the input data known as adversarial attacks. A variety of recent adversarial training procedures have been proposed to remedy this issue. Despite the success of such procedures at increasing accuracy on adversarially perturbed inputs or robust accuracy, these techniques often reduce accuracy on natural unperturbed inputs or standard accuracy. Complicating matters further the effect and trend of adversarial training procedures on standard and robust accuracy is rather counter intuitive and radically dependent on a variety of factors including the perceived form of the perturbation during training, size/quality of data, model overparameterization, etc. In this paper we focus on binary classification problems where the data is generated according to the mixture of two Gaussians with general anisotropic covariance matrices and derive a precise characterization of the standard and robust accuracy for a class of minimax adversarially trained models. We consider a general norm-based adversarial model, where the adversary can add perturbations of bounded $\ell_p$ norm to each input data, for an arbitrary $p\ge 1$. Our comprehensive analysis allows us to theoretically explain several intriguing empirical phenomena and provide a precise understanding of the role of different problem parameters on standard and robust accuracies.
Lasaft: Latent Source Attentive Frequency Transformation For Conditioned Source Separation

Recent deep-learning approaches have shown that Frequency Transformation (FT) blocks can significantly improve spectrogram-based single-source separation models by capturing frequency patterns. The goal of this paper is to extend the FT block to fit the multi-source task. We propose the Latent Source Attentive Frequency Transformation (LaSAFT) block to capture source-dependent frequency patterns. We also propose the Gated Point-wise Convolutional Modulation (GPoCM), an extension of Feature-wise Linear Modulation (FiLM), to modulate internal features. By employing these two novel methods, we extend the Conditioned-U-Net (CUNet) for multi-source separation, and the experimental results indicate that our LaSAFT and GPoCM can improve the CUNet’s performance, achieving state-of-the-art SDR performance on several MUSDB18 source separation tasks.
Neural Audio Fingerprint for High-Specific Audio Retrieval Based on Contrastive Learning

Most of existing audio fingerprinting systems have limitations to be used for high-specific audio retrieval at scale. In this work, we generate a low-dimensional representation from a short unit segment of audio, and couple this fingerprint with a fast maximum inner-product search. To this end, we present a contrastive learning framework that derives from the segment-level search objective. Each update in training uses a batch consisting of a set of pseudo labels, randomly selected original samples, and their augmented replicas. These replicas can simulate the degrading effects on original audio signals by applying small time offsets and various types of distortions, such as background noise and room/microphone impulse responses. In the segment-level search task, where the conventional audio fingerprinting systems used to fail, our system using 10x smaller storage has shown promising results. Our code and dataset are available at https://mimbres.github. io/neural-audio-fp/.
The quiescent fraction of isolated low surface brightness galaxies: observational constraints

Understanding the formation and evolution of low surface brightness galaxies (LSBGs) is critical for explaining their wide-ranging properties. However, studies of LSBGs in deep photometric surveys are often hindered by a lack of distance estimates. In this work, we present a new catalogue of 479 LSBGs, identified in deep optical imaging data from the Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP). These galaxies are found across a range of environments, from the field to groups. Many are likely to be ultra-diffuse galaxies (UDGs). We see clear evidence for a bimodal population in colour - Sersic index space, and split our sample into red and blue LSBG populations. We estimate environmental densities for a subsample of 215 sources by statistically associating them with nearby spectroscopic galaxies from the overlapping GAMA spectroscopic survey. We find that the blue LSBGs are statistically consistent with being spatially randomised with respect to local spectroscopic galaxies, implying they exist predominantly in low-density environments. However, the red LSBG population is significantly spatially correlated with local structure. We find that 26+/-5% of isolated, local LSBGs belong to the red population, which we interpret as quiescent. This indicates that high environmental density plays a dominant, but not exclusive, role in producing quiescent LSBGs. Our analysis method may prove to be very useful given the large samples of LSB galaxies without distance information expected from e.g. the Vera C. Rubin observatory (aka LSST), especially in combination with upcoming comprehensive wide field spectroscopic surveys.
Measuring cosmological distances using cluster edges as a standard ruler

The line-of-sight velocity dispersion profile of galaxy clusters exhibits a "kink" corresponding to the spatial extent of orbiting galaxies. Because the spatial extent of a cluster is correlated with the amplitude of the velocity dispersion profile, we can utilise this feature as a gravity-calibrated standard ruler. Specifically, the amplitude of the velocity dispersion data allows us to infer the physical cluster size. Consequently, observations of the angular scale of the "kink" in the profile can be translated into a distance measurement to the cluster. Assuming the relation between cluster radius and cluster velocity dispersion can be calibrated from simulations, we forecast that with existing data from the Sloan Digital Sky Survey (SDSS) we will be able to measure the Hubble constant with $3\%$ precision. Implementing our method with data from the Dark Energy Spectroscopic Instrument (DESI) will result in a $1.3\%$ measurement of the Hubble constant. Adding cosmological supernova data improves the uncertainty of the DESI measurement to $0.7\%$.
Few-Shot Adaptation of Generative Adversarial Networks

Generative Adversarial Networks (GANs) have shown remarkable performance in image synthesis tasks, but typically require a large number of training samples to achieve high-quality synthesis. This paper proposes a simple and effective method, Few-Shot GAN (FSGAN), for adapting GANs in few-shot settings (less than 100 images). FSGAN repurposes component analysis techniques and learns to adapt the singular values of the pre-trained weights while freezing the corresponding singular vectors. This provides a highly expressive parameter space for adaptation while constraining changes to the pretrained weights. We validate our method in a challenging few-shot setting of 5-100 images in the target domain. We show that our method has significant visual quality gains compared with existing GAN adaptation methods. We report qualitative and quantitative results showing the effectiveness of our method. We additionally highlight a problem for few-shot synthesis in the standard quantitative metric used by data-efficient image synthesis works. Code and additional results are available at this http URL.
High resolution weakly supervised localization architectures for medical images

In medical imaging, Class-Activation Map (CAM) serves as the main explainability tool by pointing to the region of interest. Since the localization accuracy from CAM is constrained by the resolution of the model's feature map, one may expect that segmentation models, which generally have large feature maps, would produce more accurate CAMs. However, we have found that this is not the case due to task mismatch. While segmentation models are developed for datasets with pixel-level annotation, only image-level annotation is available in most medical imaging datasets. Our experiments suggest that Global Average Pooling (GAP) and Group Normalization are the main culprits that worsen the localization accuracy of CAM. To address this issue, we propose Pyramid Localization Network (PYLON), a model for high-accuracy weakly-supervised localization that achieved 0.62 average point localization accuracy on NIH's Chest X-Ray 14 dataset, compared to 0.45 for a traditional CAM model. Source code and extended results are available at this https URL.
The effects of telemedicine on the quality of life of patients with lung cancer: a systematic review and meta-analysis

Background and aims: Lung cancer patients suffer from deterioration in their physical and psychological function, which exerts a negative influence on their quality of life (QOL). Telemedicine has been proven to be an effective intervention for patients with several chronic diseases. The aim of this systematic review and meta-analysis was to investigate the efficacy of telemedicine in improving QOL in lung cancer patients. Methods: PubMed, Cochrane Library, EMBASE, Web of Science and Scopus databases were searched for randomized controlled trials that investigated the effectiveness of telemedicine in lung cancer patients. Review Manager 5.3 and Stata 15.1 were used to perform data analysis. Results: Our meta-analysis included eight clinical trials with a total of 635 lung cancer patients. The results showed that the telemedicine group had significantly higher QOL than the usual care group [standard mean difference (SMD) 0.96, 95% confidence interval (CI) 0.29–1.63, I2 = 91%]. In addition, the telemedicine group had lower anxiety (SMD −0.44, 95% CI −0.66 to −0.23, I2 = 3%) and depression scores (SMD −0.48, 95% CI −0.91 to −0.05, I2 = 66%) than the usual care group. However, no significant differences were found in fatigue and pain outcomes between the two groups. Conclusion: Telemedicine may be an effective method of improving QOL in lung cancer patients and the further development and use of telemedicine care is recommended.
Parsec-scale properties of the radio brightest jetted AGN at z > 6

We present Director’s Discretionary Time multi-frequency observations obtained with the Jansky Very Large Array and the Very Long Baseline Array (VLBA) of the blazar PSO J030947.49+271757.31 (hereafter PSO J0309+27) at z = 6.10 ± 0.03. The milliarcsecond angular resolution of our VLBA observations at 1.5, 5, and 8.4 GHz unveils a bright one-sided jet extended for ∼500 parsecs in projection. This high-z radio-loud active galactic nucleus is resolved into multiple compact sub-components that are embedded in a more diffuse and faint radio emission that enshrouds them in a continuous jet structure. We directly derive limits on some physical parameters from observable quantities such as viewing angle and Lorentz and Doppler factors. If PSO J0309+27 is a genuine blazar, as suggested by its X-ray properties, then we find that its bulk Lorentz factor must be relatively low (lower than 5). This value would be in favour of a scenario currently proposed to reconcile the paucity of high-z blazars with current predictions. Nevertheless, we cannot exclude that PSO J0309+27 is seen under a larger viewing angle, which would imply that the X-ray emission must be enhanced, for example, by inverse Compton scattering with the cosmic microwave background. More stringent constraints on the bulk Lorentz factor in PSO J0309+27 and on these factors in the other high-z blazars are necessary to test whether their properties are intrinsically different from those of the low-z blazar population.
Scientific Claim Verification with VerT5erini

This work describes the adaptation of a pretrained sequence-to-sequence model to the task of scientific claim verification in the biomedical domain. We propose a system called VerT5erini that exploits T5 for abstract retrieval, sentence selection, and label prediction, which are three critical sub-tasks of claim verification. We evaluate our pipeline on SciFACT, a newly curated dataset that requires models to not just predict the veracity of claims but also provide relevant sentences from a corpus of scientific literature that support the prediction. Empirically, our system outperforms a strong baseline in each of the three sub-tasks. We further show VerT5erini’s ability to generalize to two new datasets of COVID-19 claims using evidence from the CORD-19 corpus.
Fast and Robust Bio-inspired Teach and Repeat Navigation

Fully autonomous mobile robots have a multitude of potential applications, but guaranteeing robust navigation performance remains an open research problem. For many tasks such as repeated infrastructure inspection, item delivery, or inventory transport, a route repeating capability can be sufficient and offers potential practical advantages over a full navigation stack. Previous teach and repeat research has achieved high performance in difficult conditions predominantly by using sophisticated, expensive sensors, and has often had high computational requirements. Biological systems, such as small animals and insects like seeing ants, offer a proof of concept that robust and generalisable navigation can be achieved with extremely limited visual systems and computing power. In this work we create a novel asynchronous formulation for teach and repeat navigation that fully utilises odometry information, paired with a correction signal driven by much more computationally lightweight visual processing than is typically required. This correction signal is also decoupled from the robot’s motor control, allowing its rate to be modulated by the available computing capacity. We evaluate this approach with extensive experimentation on two different robotic platforms, the Consequential Robotics Miro and the Clearpath Jackal robots, across navigation trials totalling more than 6000 metres in a range of challenging indoor and outdoor environments. Our approach continues to succeed when multiple state-of-the-art systems fail due to low resolution images, unreliable odometry, or lighting change, while requiring significantly less compute. We also – for the first time – demonstrate versatile cross-platform teach and repeat without changing parameters, in which we learn to navigate a route with one robot and repeat that route using a completely different robot.
Energy-Efficient Node Deployment in Heterogeneous Two-Tier Wireless Sensor Networks With Limited Communication Range

We study a heterogeneous two-tier wireless sensor network in which <inline-formula> <tex-math notation="LaTeX">$N$ </tex-math></inline-formula> heterogeneous access points (APs) collect sensing data from densely distributed sensors and then forward the data to <inline-formula> <tex-math notation="LaTeX">$M$ </tex-math></inline-formula> heterogeneous fusion centers (FCs). This heterogeneous node deployment problem is modeled as an optimization problem with the total power consumption of the network as its cost function. The necessary conditions of the optimal AP and FC node deployment are explored in this paper. We provide a variation of Voronoi Diagram as the optimal cell partition for this network and show that each AP should be placed between its connected FC and the geometric center of its cell partition. In addition, we propose a heterogeneous two-tier Lloyd algorithm to optimize the node deployment. Furthermore, we study the sensor deployment when the communication range is limited for sensors and APs. Simulation results show that our proposed algorithms outperform the existing clustering methods like Minimum Energy Routing, Agglomerative Clustering, Divisive Clustering, Particle Swarm Optimization, Relay Node placement in Double-tiered Wireless Sensor Networks, and Improved Relay Node Placement, on average.
Self-Supervised Contrastive Learning for Efficient User Satisfaction Prediction in Conversational Agents

Turn-level user satisfaction is one of the most important performance metrics for conversational agents. It can be used to monitor the agent’s performance and provide insights about defective user experiences. While end-to-end deep learning has shown promising results, having access to a large number of reliable annotated samples required by these methods remains challenging. In a large-scale conversational system, there is a growing number of newly developed skills, making the traditional data collection, annotation, and modeling process impractical due to the required annotation costs and the turnaround times. In this paper, we suggest a self-supervised contrastive learning approach that leverages the pool of unlabeled data to learn user-agent interactions. We show that the pre-trained models using the self-supervised objective are transferable to the user satisfaction prediction. In addition, we propose a novel few-shot transfer learning approach that ensures better transferability for very small sample sizes. The suggested few-shot method does not require any inner loop optimization process and is scalable to very large datasets and complex models. Based on our experiments using real data from a large-scale commercial system, the suggested approach is able to significantly reduce the required number of annotations, while improving the generalization on unseen skills.