Miracle-dz/UniFAR-FTUs huggingface dataset:
Addressing Over-Smoothing in Graph Neural Networks via Deep Supervision

Learning useful node and graph representations with graph neural networks (GNNs) is a challenging task. It is known that deep GNNs suffer from over-smoothing where, as the number of layers increases, node representations become nearly indistinguishable and model performance on the downstream task degrades significantly. To address this problem, we propose deeply-supervised GNNs (DSGNNs), i.e., GNNs enhanced with deep supervision where representations learned at all layers are used for training. We show empirically that DSGNNs are resilient to over-smoothing and can outperform competitive benchmarks on node and graph property prediction problems.
Oolong: Investigating What Makes Transfer Learning Hard with Controlled Studies

When we transfer a pretrained language model to a new language, there are many axes of variation that change at once. To disentangle the impact of different factors like syntactic similarity and vocabulary similarity, we propose a set of controlled transfer studies: we systematically transform the language of the GLUE benchmark, altering one axis of crosslingual variation at a time, and then measure the resulting drops in a pretrained model's downstream performance. We find that models can largely recover from syntactic-style shifts, but cannot recover from vocabulary misalignment and embedding matrix re-initialization, even with continued pretraining on 15 million tokens. %On the other hand, transferring to a dataset with an unaligned vocabulary is extremely hard to recover from in the low-data regime. Moreover, good-quality tokenizers in the transfer language do not make vocabulary alignment easier. Our experiments provide insights into the factors of cross-lingual transfer that researchers should most focus on when designing language transfer scenarios.
On the Use of Torque Measurement in Centroidal State Estimation

State-of-the-art legged robots are either capable of measuring torque at the output of their drive systems, or have transparent drive systems which enable the computation of joint torques from motor currents. In either case, this sensor modality is seldom used in state estimation. In this paper, we propose to use joint torque measurements to estimate the centroidal states of legged robots. To do so, we project the whole-body dynamics of a legged robot into the nullspace of the contact constraints, allowing expression of the dynamics independent of the contact forces. Using the constrained dynamics and the centroidal momentum matrix, we are able to directly relate joint torques and centroidal states dynamics. Using the resulting model as the process model of an Extended Kalman Filter (EKF), we fuse the torque measurement in the centroidal state estimation problem. Through real-world experiments on a quadruped robot executing different gaits, we demonstrate that the estimated centroidal states from our torque-based EKF drastically improve the recovery of these quantities compared to direct computation.
A sensitivity analysis approach for the causal hazard ratio in randomized and observational studies

The hazard ratio (HR) is often reported as the main causal effect when studying survival data. Despite its popularity, the HR suffers from an unclear causal interpretation. As already pointed out in the literature, there is a built‐in selection bias in the HR, because similarly to the truncation by death problem, the HR conditions on post‐treatment survival. A recently proposed alternative, inspired by the Survivor Average Causal Effect, is the causal HR, defined as the ratio between hazards across treatment groups among the study participants that would have survived regardless of their treatment assignment. We discuss the challenge in identifying the causal HR and present a sensitivity analysis identification approach in randomized controlled trials utilizing a working frailty model. We further extend our framework to adjust for potential confounders using inverse probability of treatment weighting. We present a Cox‐based and a flexible non‐parametric kernel‐based estimation under right censoring. We study the finite‐sample properties of the proposed estimation methods through simulations. We illustrate the utility of our framework using two real‐data examples.
Thompson Sampling with Unrestricted Delays

We investigate properties of Thompson Sampling in the stochastic multi-armed bandit problem with delayed feedback. In a setting with i.i. d delays, we establish to our knowledge the first regret bounds for Thompson Sampling with arbitrary delay distributions, including ones with unbounded expectation. Our bounds are qualitatively comparable to the best available bounds derived via ad-hoc algorithms, and only depend on delays via selected quantiles of the delay distributions. Furthermore, in extensive simulation experiments, we find that Thompson Sampling outperforms a number of alternative proposals, including methods specifically designed for settings with delayed feedback.
Beyond Bjøntegaard: Limits of Video Compression Performance Comparisons

For 20 years, the gold standard to evaluate the performance of video codecs is to calculate average differences between rate-distortion curves, also called the "Bjøntegaard Delta". With the help of this tool, the compression performance of codecs can be compared. In the past years, we could observe that the calculus was also deployed for other metrics than bitrate and distortion in terms of peak signal-to-noise ratio, for example other quality metrics such as video multi-method assessment fusion or hardware-dependent metrics such as the decoding energy. However, it is unclear whether the Bjøntegaard Delta is a valid way to evaluate these metrics. To this end, this paper reviews several interpolation methods and evaluates their accuracy using different performance metrics. As a result, we propose to use a novel approach based on Akima interpolation, which returns the most accurate results for a large variety of performance metrics. The approximation accuracy of this new method is determined to be below a bound of 1.5%.
Estimation of water utility compliance with trihalomethane regulations using a modelling approach

Over the last few years, the presence of chlorination by-products (CBP) in drinking water has become an issue of particular concern for utility managers. As regulations about CBP (in particular trihalomethanes – THM), are becoming more strict, water utilities will have to adjust their operation strategies to comply with new standards while maintaining a residual chlorine which ensures an acceptable microbiological quality. Complying with stricter THM standards is particularly difficult for utilities which use surface raw water and practice chlorination as the unique treatment process. In this paper, the authors explore the usefulness of predictive models as decision-making tools for drinking water managers in dealing with the THM issue. In the first part of the paper, the process of developing models of THM formation in chlorinated surface waters using two different sources of data (from bench-scale and from field-scale studies) is described. The second part of the paper focuses on the application of such models to the analysis of the feasibility of updating THM regulations in the province of Quebec (Canada). Simulations allow the estimation of the percentage of utilities, currently using chlorination as the unique treatment, which would have to upgrade treatment to comply with a new THM standard.
Optimal channel selection with discrete QCQP

Reducing the high computational cost of large convolutional neural networks is crucial when deploying the networks to resource-constrained environments. We first show the greedy approach of recent channel pruning methods ignores the inherent quadratic coupling between channels in the neighboring layers and cannot safely remove inactive weights during the pruning procedure. Furthermore, due to these inactive weights, the greedy methods cannot guarantee to satisfy the given resource constraints and deviate with the true objective. In this regard, we propose a novel channel selection method that optimally selects channels via discrete QCQP, which provably prevents any inactive weights and guarantees to meet the resource constraints tightly in terms of FLOPs, memory usage, and network size. We also propose a quadratic model that accurately estimates the actual inference time of the pruned network, which allows us to adopt inference time as a resource constraint option. Furthermore, we generalize our method to extend the selection granularity beyond channels and handle non-sequential connections. Our experiments on CIFAR-10 and ImageNet show our proposed pruning method outperforms other fixed-importance channel pruning methods on various network architectures.
Efficient Video Segmentation Models with Per-frame Inference

Most existing real-time deep models trained with each frame independently may produce inconsistent results across the temporal axis when tested on a video sequence. A few methods take the correlations in the video sequence into account, e.g., by propagating the results to the neighboring frames using optical flow, or extracting frame representations using multiframe information, which may lead to inaccurate results or unbalanced latency. In this work, we focus on improving the temporal consistency without introducing computation overhead in inference. To this end, we perform inference at each frame. Temporal consistency is achieved by learning from video frames with extra constraints during the training phase. introduced for inference. We propose several techniques to learn from the video sequence, including a temporal consistency loss and online/offline knowledge distillation methods. On the task of semantic video segmentation, weighing among accuracy, temporal smoothness, and efficiency, our proposed method outperforms keyframe based methods and a few baseline methods that are trained with each frame independently, on datasets including Cityscapes, Camvid and 300VW-Mask. We further apply our training method to video instance segmentation on YouTubeVIS and develop an application of portrait matting in video sequences, by segmenting temporally consistent instance-level trimaps across frames. Experiments show superior qualitative and quantitative results. Code is available at: https://git.io/vidseg.
Fast Consensus and Metastability in a Highly Polarized Social Network

A polarized social network is modeled as a system of interacting marked point processes with memory of variable length. Each point process indicates the successive times in which a social actor express a"favorable"or"contrary"opinion. After expressing an opinion, the social pressure on the actor is reset to 0, waiting for the group's reaction. The orientation and the rate at which an actor express an opinion is influenced by the social pressure exerted on them, modulated by a polarization coefficient. We prove that the network reaches an instantaneous but metastable consensus, when the polarization coefficient diverges.
PromDA: Prompt-based Data Augmentation for Low-Resource NLU Tasks

This paper focuses on the Data Augmentation for low-resource Natural Language Understanding (NLU) tasks. We propose Prompt-based Data Augmentation model (PromDA) which only trains small-scale Soft Prompt (i.e., a set of trainable vectors) in the frozen Pre-trained Language Models (PLMs). This avoids human effort in collecting unlabeled in-domain data and maintains the quality of generated synthetic data. In addition, PromDA generates synthetic data via two different views and filters out the low-quality data using NLU models. Experiments on four benchmarks show that synthetic data produced by PromDA successfully boost up the performance of NLU models which consistently outperform several competitive baseline models, including a state-of-the-art semi-supervised model using unlabeled in-domain data. The synthetic data from PromDA are also complementary with unlabeled in-domain data. The NLU models can be further improved when they are combined for training.
Learning Invariant Weights in Neural Networks

Assumptions about invariances or symmetries in data can significantly increase the predictive power of statistical models. Many commonly used models in machine learning are constraint to respect certain symmetries in the data, such as translation equivariance in convolutional neural networks, and incorporation of new symmetry types is actively being studied. Yet, efforts to learn such invariances from the data itself remains an open research problem. It has been shown that marginal likelihood offers a principled way to learn invariances in Gaussian Processes. We propose a weight-space equivalent to this approach, by minimizing a lower bound on the marginal likelihood to learn invariances in neural networks resulting in naturally higher performing models.
Construction of Large-Scale Misinformation Labeled Datasets from Social Media Discourse using Label Refinement

Malicious accounts spreading misinformation has led to widespread false and misleading narratives in recent times, especially during the COVID-19 pandemic, and social media platforms struggle to eliminate these contents rapidly. This is because adapting to new domains requires human intensive fact-checking that is slow and difficult to scale. To address this challenge, we propose to leverage news-source credibility labels as weak labels for social media posts and propose model-guided refinement of labels to construct large-scale, diverse misinformation labeled datasets in new domains. The weak labels can be inaccurate at the article or social media post level where the stance of the user does not align with the news source or article credibility. We propose a framework to use a detection model self-trained on the initial weak labels with uncertainty sampling based on entropy in predictions of the model to identify potentially inaccurate labels and correct for them using self-supervision or relabeling. The framework will incorporate social context of the post in terms of the community of its associated user for surfacing inaccurate labels towards building a large-scale dataset with minimum human effort. To provide labeled datasets with distinction of misleading narratives where information might be missing significant context or has inaccurate ancillary details, the proposed framework will use the few labeled samples as class prototypes to separate high confidence samples into false, unproven, mixture, mostly false, mostly true, true, and debunk information. The approach is demonstrated for providing a large-scale misinformation dataset on COVID-19 vaccines.
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?

Large language models (LMs) are able to in-context learn—perform a new task via inference alone by conditioning on a few input-label pairs (demonstrations) and making predictions for new inputs. However, there has been little understanding of how the model learns and which aspects of the demonstrations contribute to end task performance. In this paper, we show that ground truth demonstrations are in fact not required—randomly replacing labels in the demonstrations barely hurts performance on a range of classification and multi-choce tasks, consistently over 12 different models including GPT-3. Instead, we find that other aspects of the demonstrations are the key drivers of endtask performance, including the fact that they provide a few examples of (1) the label space, (2) the distribution of the input text, and (3) the overall format of the sequence. Together, our analysis provides a new way of understanding how and why in-context learning works, while opening up new questions about how much can be learned from large language models through inference alone.
Boosting distributional copula regression

Capturing complex dependence structures between outcome variables (e.g., study endpoints) is of high relevance in contemporary biomedical data problems and medical research. Distributional copula regression provides a flexible tool to model the joint distribution of multiple outcome variables by disentangling the marginal response distributions and their dependence structure. In a regression setup, each parameter of the copula model, that is, the marginal distribution parameters and the copula dependence parameters, can be related to covariates via structured additive predictors. We propose a framework to fit distributional copula regression via model‐based boosting, which is a modern estimation technique that incorporates useful features like an intrinsic variable selection mechanism, parameter shrinkage and the capability to fit regression models in high‐dimensional data setting, that is, situations with more covariates than observations. Thus, model‐based boosting does not only complement existing Bayesian and maximum‐likelihood based estimation frameworks for this model class but rather enables unique intrinsic mechanisms that can be helpful in many applied problems. The performance of our boosting algorithm for copula regression models with continuous margins is evaluated in simulation studies that cover low‐ and high‐dimensional data settings and situations with and without dependence between the responses. Moreover, distributional copula boosting is used to jointly analyze and predict the length and the weight of newborns conditional on sonographic measurements of the fetus before delivery together with other clinical variables.
Take the A-Metric: Interpretations of Some Known Solutions of Einstein’s Vacuum Field Equations

In this work, we present a new interpretation of the only static vacuum solution of Einstein’s field equations with planar symmetry, the Taub solution. This solution is a member of the AIII class of metrics, along with the type D Kasner solution. Various interpretations of these solutions have been put forward previously in the literature, however, some of these interpretations have suspect features and are not generally considered physical. Using a simple mathematical analysis, we show that a novel interpretation of the Taub solution is possible and that it naturally emerges from the radial, near-singularity limit of negative-mass Schwarzschild spacetime. A new, more transparent derivation is also given, showing that the type D Kasner metric can be interpreted as a region of spacetime deep within a positive-mass Schwarzschild black hole. The dual nature of this class of A-metrics is thereby demonstrated.
Neutron stars and phase diagram in a hard-wall AdS/QCD model

We study the phase diagram of (large-$N_c$) QCD using a simplistic holographic hard-wall model with a dynamical scalar field and a homogeneous Ansatz representing a smeared instanton/baryon density. The resulting phase diagram is qualitatively consistent with expectations, including a mesonic, baryonic, quarkyonic, and quark-gluon plasma phase. As in other holographic models, we also find a baryonic popcorn transition, which appears at large chemical potential as a crossover. We then evaluate the nuclear matter equation of state, which turns out to be rather stiff with a large peaked sound velocity above the conformal limit, construct corresponding neutron stars using the TOV equations, and finally use a full numerical gravity/hydrodynamics computation to extract the gravitational wave signal of neutron star mergers.
Designing Social VR: A Collection of Design Choices Across Commercial and Research Applications

Social VR has experienced tremendous growth in the commercial space recently as an emerging technology for rich interactions themed around leisure, work, and relationship building. As a result, the state of social VR application design has become rapidly obfuscated, which complicates identi ﬁ cation of design trends and uncommon features that could inform future design, and hinders inclusion of new voices in this design space. To help address this problem, we present a taxonomy of social VR application design choices as informed by 44 commercial and prototypical applications. Our taxonomy was informed by multiple discovery strategies including literature review, search of VR-themed subreddits, and autobiographical landscape research. Th e taxonomy elucidates three design areas—the self, interaction, and the environment— and organizes 10 feature categories and 38 feature variations. Th e paper re ﬂ ects on how the taxonomy can support involvement of new designers, and considers possible expansion and use by researchers.
Quantum fluctuations, particles and entanglement: A discussion towards the solution of the quantum measurement problems

The quantum measurement problems are revisited from a new perspective. One of the main ideas of this work is that the basic entities of our world are various types of particles, elementary or composite. It follows that each elementary process, hence each measurement process at its core, is a spacetime, pointlike, event. Another key idea is that, when a microsystem ψ gets into contact with the experimental device, factorization of ψ rapidly fails and entangled mixed states appear. The wave functions for the microsystem-apparatus coupled systems for diﬀerent measurement outcomes then lack overlapping spacetime support. It means that the aftermath of each measurement is a single term in the sum: a “wave-function collapse”. Our discussion leading to a diagonal density matrix, ρ = diag( | c 1 | 2 , . . . , | c n | 2 , . . . ) shows how the information encoded in the wave function | ψ (cid:105) = (cid:80) n c n | n (cid:105) gets tran-scribed, via entanglement with the experimental device and environment, into the relative frequencies P n = | c n | 2 for various experimental results F = f n . These results represent new, signiﬁcant steps towards ﬁlling in the logical gaps in the standard interpretation based on Born’s rule, and replacing it with a more natural one. Accepting objective reality of quantum ﬂuctuations, independent of any experiments, and independently of human presence, one renounces the idea that in a fundamental, complete theory of Nature the result of each single experiment must necessarily be predictable.
Pseudomonas aeruginosa Invasion by Permeability, and Prevents Epithelial Cell Stiffness, Decreases Transepithelial Cathelicidin LL-37 Increases Lung Epithelial

In addition to its antibacterial activity, the cathelicidin-derived LL-37 peptide induces multiple immunomodulatory effects on host cells. Atomic force microscopy, F-actin staining with phalloidin, passage of FITC-conjugated dextran through a monolayer of lung epithelial cells, and assessment of bacterial outgrowth from cells subjected to Pseudomonas aeruginosa infection were used to determine LL-37’s effect on epithelial cell mechanical properties, permeability, and bacteria uptake. A concentration-dependent increase in stiffness and F-actin content in the cortical region of A549 cells and primary human lung epithelial cells was observed after treatment with LL-37 (0.5–5 m M), sphingosine 1-phosphate (1 m M), or LPS (1 m g/ml) or infection with PAO1 bacteria. Other cationic peptides, such as RK-31, KR-20, or WLBU2, and the antibacterial cationic steroid CSA-13 did not reproduce the effect of LL-37. A549 cell pretreatment with WRW4, an antagonist of the transmembrane formyl peptide receptor-like 1 protein atten-uated LL-37’s ability to increase cell stiffness. The LL-37–mediated increase in cell stiffness was accompanied by a decrease in permeability and P. aeruginosa uptake by a conﬂuent monolayer of polarized normal human bronchial epithelial cells. These results suggested that the antibacterial effect of LL-37 involves an LL-37–dependent increase in cell stiffness that prevents epithelial invasion by bacteria. The Journal of Immunology