Miracle-dz/UniFAR-FTUs huggingface dataset:
CCRL: Contrastive Cell Representation Learning

Cell identification within the H&E slides is an essential prerequisite that can pave the way towards further pathology analyses including tissue classification, cancer grading, and phenotype prediction. However, performing such a task using deep learning techniques requires a large cell-level annotated dataset. Although previous studies have investigated the performance of contrastive self-supervised methods in tissue classification, the utility of this class of algorithms in cell identification and clustering is still unknown. In this work, we investigated the utility of Self-Supervised Learning (SSL) in cell clustering by proposing the Contrastive Cell Representation Learning (CCRL) model. Through comprehensive comparisons, we show that this model can outperform all currently available cell clustering models by a large margin across two datasets from different tissue types. More interestingly, the results show that our proposed model worked well with a few number of cell categories while the utility of SSL models has been mainly shown in the context of natural image datasets with large numbers of classes (e.g., ImageNet). The unsupervised representation learning approach proposed in this research eliminates the time-consuming step of data annotation in cell classification tasks, which enables us to train our model on a much larger dataset compared to previous methods. Therefore, considering the promising outcome, this approach can open a new avenue to automatic cell representation learning.
A Census of the 32 Ori Association with Gaia

I have used high-precision photometry and astrometry from the third data release of Gaia (DR3) to identify candidate members of the 32 Ori association. Spectral types and radial velocities have been measured for subsets of the candidates using new and archival spectra. For the candidates that have radial velocity measurements, I have used UVW velocities to further constrain their membership, arriving at a final catalog of 169 candidates. I estimate that the completeness of the survey is ∼90% for spectral types of ≲M7 (≳0.06 M ⊙). The histogram of spectral types for the 32 Ori candidates exhibits a maximum at M5 (∼0.15 M ⊙), resembling the distributions measured for other young clusters and associations in the solar neighborhood. The available UVW velocities indicate that the association is expanding, but they do not produce a well-defined kinematic age. Based on their sequences of low-mass stars in color–magnitude diagrams, the 32 Ori association and Upper Centaurus-Lupus/Lower Centaurus-Crux (UCL/LCC) are coeval to within ±1.2 Myr, and they are younger than the β Pic moving group by ∼3 Myr, which agrees with results from previous analysis based on the second data release of Gaia. Finally, I have used mid-IR photometry from the Wide-field Infrared Survey Explorer to check for excess emission from circumstellar disks among the 32 Ori candidates. Disks are detected for 18 candidates, half of which are reported for the first time in this work. The fraction of candidates at ≤M6 that have full, transitional, or evolved disks is 10/149=0.07−0.02+0.03 , which is consistent with the value for UCL/LCC.
Towards Interpretable Sleep Stage Classification Using Cross-Modal Transformers

Accurate sleep stage classification is significant for sleep health assessment. In recent years, several machine-learning based sleep staging algorithms have been developed , and in particular, deep-learning based algorithms have achieved performance on par with human annotation. Despite improved performance, a limitation of most deep-learning based algorithms is their black-box behavior, which have limited their use in clinical settings. Here, we propose a cross-modal transformer, which is a transformer-based method for sleep stage classification. The proposed cross-modal transformer consists of a novel cross-modal transformer encoder architecture along with a multi-scale one-dimensional convolutional neural network for automatic representation learning. Our method outperforms the state-of-the-art methods and eliminates the black-box behavior of deep-learning models by utilizing the interpretability aspect of the attention modules. Furthermore, our method provides considerable reductions in the number of parameters and training time compared to the state-of-the-art methods. Our code is available at https://github.com/Jathurshan0330/Cross-Modal-Transformer. A demo of our work can be found at https://bit.ly/Cross_modal_transformer_demo.
Operads as double functors

It is shown how double categories provide a direct abstract approach to coloured operads; namely, product-preserving normal lax functors from (Pb C)^op (the opposite of the double category of pullback squares in C) to Cat (the double category of functors and profunctors) can be seen as generalized operads, the standard ones arising when C = Set_f. In this context, generalized symmetric monoidal categories are considered, in particular those arising from indexed categories with sums or products.
Differentially Private Hypothesis Testing With the Subsampled and Aggregated Randomized Response Mechanism

Randomized response is one of the oldest and most well-known methods for analyzing confidential data. However, its utility for differentially private hypothesis testing is limited because it cannot achieve high privacy levels and low type I error rates simultaneously. In this article, we show how to overcome this issue with the subsample and aggregate technique. The result is a general-purpose method that can be used for both frequentist and Bayesian testing. {{We illustrate the performance of our proposal in three scenarios: goodness-of-fit testing for linear regression models, nonparametric testing of a location parameter with the Wilcoxon test, and the nonparametric Kruskal-Wallis test.
Can a latent Hawkes process be used for epidemiological modelling?

Understanding the spread of COVID-19 has been the subject of numerous studies, highlighting the significance of reliable epidemic models. Here, we introduce a novel epidemic model using a latent Hawkes process with temporal covariates for modelling the infections. Unlike other models, we model the reported cases via a probability distribution driven by the underlying Hawkes process. Modelling the infections via a Hawkes process allows us to estimate by whom an infected individual was infected. We propose a Kernel Density Particle Filter (KDPF) for inference of both latent cases and reproduction number and for predicting the new cases in the near future. The computational effort is proportional to the number of infections making it possible to use particle filter type algorithms, such as the KDPF. We demonstrate the performance of the proposed algorithm on synthetic data sets and COVID-19 reported cases in various local authorities in the UK, and benchmark our model to alternative approaches.
Global dynamics for a class of discrete fractional epidemic model with reaction-diffusion

In recent years, discrete fractional epidemic models with reaction-diffusion have become increasingly popular in the literature, not only for its necessity of numerical simulation, but also for its defined physical processes. In this paper, by second order central difference scheme and L1 nonstandard finite difference scheme, a discrete counterpart of time-fractional reaction-diffusion epidemic model with generalized incidence rate is considered. More importantly, the main idea in choosing an nonstandard finite difference scheme is to obtain unconditionally positivity in the proposed system, which leads to the proposal of the discrete epidemic model with time delay. Furthermore, the global properties of the proposed discrete system are studied, including the global boundedness of positive solutions, the existence and the global stability of equilibrium points, which are consistent with the corresponding continuous systems. Meanwhile, it shows that L1 nonstandard finite difference scheme and second order central difference scheme can keep the properties of the corresponding continuous system well. It is worth noting that, different from discrete epidemic model with integer-order, the memory Lyapunov function is constructed in this paper, which depends on the previous historical information of the proposed system. This is consistent with the non-local property of Caputo fractional derivatives. Finally, numerical results are given to verify the theoretical results.
Identification of the energy contributions associated with wall-attached eddies and very-large-scale motions in the near-neutral atmospheric surface layer through wind LiDAR measurements

Abstract Recent works on wall-bounded flows have corroborated the coexistence of wall-attached eddies, whose statistical features are predicted through Townsend's attached-eddy hypothesis (AEH), and very-large-scale motions (VLSMs). Furthermore, it has been shown that the presence of wall-attached eddies within the logarithmic layer is linked to the appearance of an inverse-power-law region in the streamwise velocity energy spectra, upon significant separation between outer and viscous scales. In this work, a near-neutral atmospheric surface layer is probed with wind light detection and ranging to investigate the contributions to the streamwise velocity energy associated with wall-attached eddies and VLSMs for a very-high-Reynolds-number boundary layer. Energy and linear coherence spectra (LCS) of the streamwise velocity are interrogated to identify the spectral boundaries associated with eddies of different typologies. Inspired by the AEH, an analytical model for the LCS associated with wall-attached eddies is formulated. The experimental results show that the identification of the wall-attached-eddy energy contribution through the analysis of the energy spectra leads to an underestimate of the associated spectral range, maximum height attained and turbulence intensity. This feature is due to the overlap of the energy associated with VLSMs obscuring the inverse-power-law region. The LCS analysis estimates wall-attached eddies with a streamwise/wall-normal ratio of about 14.3 attaining a height of about 30 % of the outer scale of turbulence.
Analysis of Impact of Emotions on Target Speech Extraction and Speech Separation

Recently, the performance of blind speech separation (BSS) and target speech extraction (TSE) has greatly progressed. Most works, however, focus on relatively well-controlled conditions using, e.g., read speech. The performance may degrade in more realistic situations. One of the factors causing such degradation may be intrinsic speaker variability, such as emotions, occurring commonly in realistic speech. In this paper, we investigate the influence of emotions on TSE and BSS. We create a new test dataset of emotional mixtures for the evaluation of TSE and BSS. This dataset combines LibriSpeech and Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS). Through controlled experiments, we can analyze the impact of different emotions on the performance of BSS and TSE. We observe that BSS is relatively robust to emotions, while TSE, which requires identifying and extracting the speech of a tar-get speaker, is much more sensitive to emotions. On comparative speaker verification experiments we show that identifying the tar-get speaker may be particularly challenging when dealing with emotional speech. Using our findings, we outline potential future directions that could improve the robustness of BSS and TSE systems toward emotional speech.
Emergent Metric-like States of Active Particles with Metric-free Polar Alignment

We study a model of self-propelled particles interacting with their k nearest neighbors through polar alignment. By exploring its phase space as a function of two nondimensional parameters (alignment strength g and Peclet number Pe), we identify two distinct order-disorder transitions. One is continuous, occurs at a low critical g value independent of Pe, and resembles a mean-ﬁeld transition with no density-order coupling. The other is discontinuous, depends on a combined control parameter involving g and Pe, and results from the formation of small, dense, highly persistent clusters of particles that follow metric-like dynamics. These dense clusters form at a critical value of the combined control parameter Pe /g α , with α ≈ 1 . 5, which appears to be valid for diﬀerent alignment-based models. Our study shows that models of active particles with metric-free interactions can produce characteristic length-scales and self-organize into metric-like collective states that undergo metric-like transitions.
Kinematics and physical properties of a highly filamentary Infrared Dark Cloud

This thesis contains a detailed study of the kinematics and physical properties of a potential site of massive star formation; the IRDC G035.39-00.33. The gas kinematics are first of all investigated using high-spectral resolution and high-sensitivity data from the IRAM 30 m telescope. The primary focus of this work is the J = 1 → 0 transition of both N2H+ and C18O, as well as N2H+ (3 − 2). Dense gas is found to be extended over ∼ 3 pc scales within G035.39-00.33. The C18 O observations confirm the presence of at least three morphologically distinct filamentary components. It is speculated that the merging of filaments may be responsible for the formation of localised density enhancements at their interface; the potential sites for massive star and star-cluster formation. The kinematic properties of the dense gas are then probed at high-angular resolution, using observations of N2H+ (1−0) from the Plateau de Bure Interferometer. It is revealed that the dense gas of G035.39-00.33 is organised into a complex network of mildly supersonic filaments separated in velocity by < 1 km s−1 . Whilst global velocity gradients throughout each filament are small, there is evidence for dynamic processes on local scales. This suggests that the kinematics are influenced by the dense (and in some cases, starless) cores. The physical properties of the embedded core population are derived in the final study of this thesis. A total of 14 continuum peaks are identified, representative of the pre- and protostellar core population covering two main clumps within G035.39-00.33. The derived core masses are found to be between 2.4-12.3 solar masses, with sizes and densities between 0.03-0.07 pc and 1.6×10^5-7.3×10^5 cm^-3, respectively. Some of the cores exhibit irregular boundaries, which may imply the presence of unresolved sub-structure. Although the dynamical state of each core is dependent on both its geometry and density profile (which are both sources of uncertainty) it is found that many of the identified cores are unstable to collapse. Cores which are well represented by monolithic, centrally condensed structures, exhibiting low virial parameters and many Jeans masses, are good candidates for the progenitors of intermediate-to-high-mass stars. Within the selected area of G035.39-00.33, two of the identified cores meet this criteria.
Teacher Guided Training: An Efficient Framework for Knowledge Transfer

The remarkable performance gains realized by large pretrained models, e.g., GPT-3, hinge on the massive amounts of data they are exposed to during training. Analogously, distilling such large models to compact models for efficient deployment also necessitates a large amount of (labeled or unlabeled) training data. In this paper, we propose the teacher-guided training (TGT) framework for training a high-quality compact model that leverages the knowledge acquired by pretrained generative models, while obviating the need to go through a large volume of data. TGT exploits the fact that the teacher has acquired a good representation of the underlying data domain, which typically corresponds to a much lower dimensional manifold than the input space. Furthermore, we can use the teacher to explore input space more efficiently through sampling or gradient-based methods; thus, making TGT especially attractive for limited data or long-tail settings. We formally capture this benefit of proposed data-domain exploration in our generalization bounds. We find that TGT can improve accuracy on several image classification benchmarks as well as a range of text classification and retrieval tasks.
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale

Large language models have been widely adopted but require significant GPU memory for inference. We develop a procedure for Int8 matrix multiplication for feed-forward and attention projection layers in transformers, which cut the memory needed for inference by half while retaining full precision performance. With our method, a 175B parameter 16/32-bit checkpoint can be loaded, converted to Int8, and used immediately without performance degradation. This is made possible by understanding and working around properties of highly systematic emergent features in transformer language models that dominate attention and transformer predictive performance. To cope with these features, we develop a two-part quantization procedure, LLM.int8(). We first use vector-wise quantization with separate normalization constants for each inner product in the matrix multiplication, to quantize most of the features. However, for the emergent outliers, we also include a new mixed-precision decomposition scheme, which isolates the outlier feature dimensions into a 16-bit matrix multiplication while still more than 99.9% of values are multiplied in 8-bit. Using LLM.int8(), we show empirically it is possible to perform inference in LLMs with up to 175B parameters without any performance degradation. This result makes such models much more accessible, for example making it possible to use OPT-175B/BLOOM on a single server with consumer GPUs. We open-source our software.
Learning to Infer Counterfactuals: Meta-Learning for Estimating Multiple Imbalanced Treatment Effects

We regularly consider answering counterfactual questions in practice, such as"Would people with diabetes take a turn for the better had they choose another medication? ". Observational studies are growing in significance in answering such questions due to their widespread accumulation and comparatively easier acquisition than Randomized Control Trials (RCTs). Recently, some works have introduced representation learning and domain adaptation into counterfactual inference. However, most current works focus on the setting of binary treatments. None of them considers that different treatments' sample sizes are imbalanced, especially data examples in some treatment groups are relatively limited due to inherent user preference. In this paper, we design a new algorithmic framework for counterfactual inference, which brings an idea from Meta-learning for Estimating Individual Treatment Effects (MetaITE) to fill the above research gaps, especially considering multiple imbalanced treatments. Specifically, we regard data episodes among treatment groups in counterfactual inference as meta-learning tasks. We train a meta-learner from a set of source treatment groups with sufficient samples and update the model by gradient descent with limited samples in target treatment. Moreover, we introduce two complementary losses. One is the supervised loss on multiple source treatments. The other loss which aligns latent distributions among various treatment groups is proposed to reduce the discrepancy. We perform experiments on two real-world datasets to evaluate inference accuracy and generalization ability. Experimental results demonstrate that the model MetaITE matches/outperforms state-of-the-art methods.
Untangling Force-Directed Layouts Using Persistent Homology

Force-directed layouts belong to a popular class of methods used to position nodes in a node-link diagram. However, they typically lack direct consideration of global structures, which can result in visual clutter and the overlap of unrelated structures. In this paper, we use the principles of persistent homology to untangle force-directed layouts thus mitigating these issues. First, we devise a new method to use 0-dimensional persistent homology to efficiently generate an initial graph layout. The approach results in faster convergence and better quality graph layouts. Second, we provide a new definition and an efficient algorithm for 1-dimensional persistent homology features (i.e., tunnels/cycles) on graphs. We provide users the ability to interact with the 1-dimensional features by highlighting them and adding cycle-emphasizing forces to the layout. Finally, we evaluate our approach with 32 synthetic and real-world graphs by computing various metrics, e.g., co-ranking, edge crossing, etc., to demonstrate the efficacy of our proposed method.
Scaling and the Universality of Function Diversity Across Human Organizations

Function diversity, namely, the range of tasks individuals can perform, is essential to productive organizations. This concept has been studied in disparate discipline contexts, while general patterns and mechanisms remain unclear. Here, we first analyze over five thousand observations of top-down organizations—US federal agencies, Norwegian companies, and US universities, and find that the number of distinct functions scales with organizational size, approximately as a power law with an exponent of 1/2. Further, we find common patterns in the distribution of function abundance within organizations. This universality suggests that human organizations, despite differences in their purpose, structure, and culture, may share common mechanisms for creating specializations. Additionally, we find that cities—bottom-up organizations—differ from top-down organizations and exhibit logarithmic scaling. We discuss potential avenues for modeling the mechanisms for these observations using history-dependent random processes, and offer several criteria for model selection.
DM-NeRF: 3D Scene Geometry Decomposition and Manipulation from 2D Images

In this paper, we study the problem of 3D scene geometry decomposition and manipulation from 2D views. By leveraging the recent implicit neural representation techniques, particularly the appealing neural radiance fields, we introduce an object field component to learn unique codes for all individual objects in 3D space only from 2D supervision. The key to this component is a series of carefully designed loss functions to enable every 3D point, especially in non-occupied space, to be effectively optimized even without 3D labels. In addition, we introduce an inverse query algorithm to freely manipulate any specified 3D object shape in the learned scene representation. Notably, our manipulation algorithm can explicitly tackle key issues such as object collisions and visual occlusions. Our method, called DM-NeRF, is among the first to simultaneously reconstruct, decompose, manipulate and render complex 3D scenes in a single pipeline. Extensive experiments on three datasets clearly show that our method can accurately decompose all 3D objects from 2D views, allowing any interested object to be freely manipulated in 3D space such as translation, rotation, size adjustment, and deformation.