Miracle-dz/UniFAR-FTUs huggingface dataset:
Edge Representation Learning with Hypergraphs

Graph neural networks have recently achieved remarkable success in representing graph-structured data, with rapid progress in both the node embedding and graph pooling methods. Yet, they mostly focus on capturing information from the nodes considering their connectivity, and not much work has been done in representing the edges, which are essential components of a graph. However, for tasks such as graph reconstruction and generation, as well as graph classification tasks for which the edges are important for discrimination, accurately representing edges of a given graph is crucial to the success of the graph representation learning. To this end, we propose a novel edge representation learning framework based on Dual Hypergraph Transformation (DHT), which transforms the edges of a graph into the nodes of a hypergraph. This dual hypergraph construction allows us to apply message-passing techniques for node representations to edges. After obtaining edge representations from the hypergraphs, we then cluster or drop edges to obtain holistic graph-level edge representations. We validate our edge representation learning method with hypergraphs on diverse graph datasets for graph representation and generation performance, on which our method largely outperforms existing graph representation learning methods. Moreover, our edge representation learning and pooling method also largely outperforms state-of-the-art graph pooling methods on graph classification, not only because of its accurate edge representation learning, but also due to its lossless compression of the nodes and removal of irrelevant edges for effective message-passing.
Asymmetric Field Photovoltaic Cell of Neutral Atoms

Wenxi Lai, Jinyan Niu, Yu-Quan Ma, and W. M. Liu 1 School of Applied Science, Beijing Information Science and Technology University, Beijing 100192, China 2 School of Science, Inner Mongolia University of Science and Technology, Baotou, 014010, China 3 Beijing National Laboratory for Condensed Matter Physics, Institute of Physics, Chinese Academy of Sciences, Beijing 100190, China and 4 School of Physical Sciences, University of Chinese Academy of Sciences, Beijing 100190, China
Automated Onychomycosis Detection Using Deep Neural Networks

—Clinical dermatology, still relies heavily on manual introspection of fungi within a Potassium Hydroxide (KOH) solution using a brightﬁeld microscope. However, this method takes a long time, is based on the experience of the clinician, and has a low accuracy. With the increase of neural network applications in the ﬁeld of clinical microscopy it is now possible to automate such manual processes increasing both efﬁciency and accuracy. This study presents a deep neural network structure that enables the rapid solutions for these problems and can perform automatic fungi detection in grayscale images without colorants. Microscopic images of 81 fungi and 235 ceratine were collected. Then, smaller patches were extracted containing 2062 fungi and 2142 ceratine. In order to detect fungus and ceratine, two models were created one of which was a custom neural network and the other was based on the VGG16 architecture. The developed custom model had 99.84% accuracy, and an area under the curve (AUC) value of 1.00, while the VGG16 model had 98.89% accuracy and an AUC value of 0.99. However, average accuracy and AUC value of clinicians is 72.8% and 0.87 respectively. This deep learning model allows the development of an automated system that can detect fungi within microscopic images.
An Experimental Analysis on Drone-Mounted Access Points for Improved Latency-Reliability

The anticipated densification of contemporary communications infrastructure expects the use of drone small cells (DSCs). Thus, we experimentally evaluate the capability of providing local and personalized coverage with a drone mounted Wi-Fi access point that uses the nearby LTE infrastructure as a backhaul in areas with mixed line of sight (LoS) and Non-LoS (NLoS) links to the local cellular infrastructure. To assess the potential of DSCs for reliable and low latency communication of outdoor users, we measure the channel quality and the total round trip latency of the system. For a drone following the ground user, the DSC-provided network extends the coverage for an extra 6.4% when compared to the classical LTE-direct link. Moreover, the DSC setup provides latencies that are consistently smaller than 50 ms for 95% of the experiment. Within the coverage of the LTE-direct connection, we observed a latency ceiling of 120 ms for 95% reliability of the LTE-direct connection. The highest latency observed for the DSC system was 1200 ms, while the LTE-direct link never exceeded 500 ms. As such, DSC setups are not only essential in NLoS situations, but consistently improve the latency of users in outdoor scenarios.
What Can Unsupervised Machine Translation Contribute to High-Resource Language Pairs?

Whereas existing literature on unsupervised machine translation (MT) focuses on exploiting unsupervised techniques for low-resource language pairs where bilingual training data is scare or unavailable, we investigate whether unsupervised MT can also improve translation quality of high-resource language pairs where sufﬁcient bitext does exist. We compare the style of correct translations generated by either supervised or unsupervised MT and ﬁnd that the unsupervised output is less monotonic and more natural than supervised output. We demonstrate a way to combine the beneﬁts of unsupervised and supervised MT into a single system, resulting in better human evaluation of quality and ﬂuency. Our results open the door to discussions about the potential contributions of unsupervised MT in high-resource settings, and how supervised and unsupervised systems might be mutually-beneﬁcial.
Bayesian inference with the l1-ball prior: solving combinatorial problems with exact zeros

The l1-regularisation is very popular in high-dimensional statistics—it changes a combinatorial problem of choosing which subset of the parameter is zero, into a simple continuous optimisation. Using a continuous prior concentrated near zero, the Bayesian counterparts are successful in quantifying the uncertainty in the variable selection problems; nevertheless, the lack of exact zeros makes it difficult for broader problems such as change-point detection and rank selection. Inspired by the duality of the l1-regularisation as a constraint onto an l1-ball, we propose a new prior by projecting a continuous distribution onto the l1-ball. This creates a positive probability on the ball boundary, which contains both continuous elements and exact zeros. Unlike the spike-and-slab prior, this l1-ball projection is continuous and differentiable almost surely, making the posterior estimation amenable to the Hamiltonian Monte Carlo algorithm. We examine the properties, such as the volume change due to the projection, the connection to the combinatorial prior, the minimax concentration rate in the linear problem. We demonstrate the usefulness of exact zeros that simplify the combinatorial problems, such as the change-point detection in time series, the dimension selection of mixture models, and the low-rank plus-sparse change detection in medical images.
Machine Learning-Based Lie Detector Applied to a Novel Annotated Game Dataset

Lie detection is considered a concern for everyone in their day-to-day life, given its impact on human interactions. Thus, people normally pay attention to both what their interlocutors are saying and to their visual appearance, including the face, to find any signs that indicate whether or not the person is telling the truth. While automatic lie detection may help us to understand these lying characteristics, current systems are still fairly limited, partly due to lack of adequate datasets to evaluate their performance in realistic scenarios. In this work, we collect an annotated dataset of facial images, comprising both 2D and 3D information of several participants during a card game that encourages players to lie. Using our collected dataset, we evaluate several types of machine learning-based lie detectors in terms of their generalization, in person-specific and cross-application experiments. We first extract both handcrafted and deep learning-based features as relevant visual inputs, then pass them into multiple types of classifier to predict respective lie/non-lie labels. Subsequently, we use several metrics to judge the models’ accuracy based on the models predictions and ground truth. In our experiment, we show that models based on deep learning achieve the highest accuracy, reaching up to 57% for the generalization task and 63% when applied to detect the lie to a single participant. We further highlight the limitation of the deep learning-based lie detector when dealing with cross-application lie detection tasks. Finally, this analysis along the proposed datasets would potentially be useful not only from the perspective of computational systems perspective (e.g., improving current automatic lie prediction accuracy), but also for other relevant application fields, such as health practitioners in general medical counselings, education in academic settings or finance in the banking sector, where close inspections and understandings of the actual intentions of individuals can be very important.
A Generative Model for Raw Audio Using Transformer Architectures

This paper proposes a novel way of doing audio synthesis at the waveform level using Transformer architectures. We propose a deep neural network for generating waveforms, similar to wavenet [1]. This is fully probabilistic, auto-regressive, and causal, i.e. each sample generated depends on only the previously observed samples. Our approach outperforms a widely used wavenet architecture by up to 9% on a similar dataset for predicting the next step. Using the attention mechanism, we enable the architecture to learn which audio samples are important for the prediction of the future sample. We show how causal transformer generative models can be used for raw waveform synthesis. We also show that this performance can be improved by another 2% by conditioning samples over a wider context. The flexibility of the current model to synthesize audio from latent representations suggests a large number of potential applications. The novel approach of using generative transformer architectures for raw audio synthesis is, however, still far away from generating any meaningful music similar to wavenet, without using latent codes/meta-data to aid the generation process.
IMS’ Systems for the IWSLT 2021 Low-Resource Speech Translation Task

This paper describes the submission to the IWSLT 2021 Low-Resource Speech Translation Shared Task by IMS team. We utilize state-of-the-art models combined with several data augmentation, multi-task and transfer learning approaches for the automatic speech recognition (ASR) and machine translation (MT) steps of our cascaded system. Moreover, we also explore the feasibility of a full end-to-end speech translation (ST) model in the case of very constrained amount of ground truth labeled data. Our best system achieves the best performance among all submitted systems for Congolese Swahili to English and French with BLEU scores 7.7 and 13.7 respectively, and the second best result for Coastal Swahili to English with BLEU score 14.9.
From kinetic to fluid models of liquid crystals by the moment method

This paper deals with the convergence of the Doi-Navier-Stokes model of liquid crystals to the Ericksen-Leslie model in the limit of the Deborah number tending to zero. While the literature has investigated this problem by means of the Hilbert expansion method, we develop the moment method, i.e. a method that exploits conservation relations obeyed by the collision operator. These are non-classical conservation relations which are associated with a new concept, that of Generalized Collision Invariant (GCI). In this paper, we develop the GCI concept and relate it to geometrical and analytical structures of the collision operator. Then, the derivation of the limit model using the GCI is performed in an arbitrary number of spatial dimensions and with non-constant and non-uniform polymer density. This non-uniformity generates new terms in the Ericksen-Leslie model.
ResViT: Residual Vision Transformers for Multimodal Medical Image Synthesis

Generative adversarial models with convolutional neural network (CNN) backbones have recently been established as state-of-the-art in numerous medical image synthesis tasks. However, CNNs are designed to perform local processing with compact filters, and this inductive bias compromises learning of contextual features. Here, we propose a novel generative adversarial approach for medical image synthesis, ResViT, that leverages the contextual sensitivity of vision transformers along with the precision of convolution operators and realism of adversarial learning. ResViT’s generator employs a central bottleneck comprising novel aggregated residual transformer (ART) blocks that synergistically combine residual convolutional and transformer modules. Residual connections in ART blocks promote diversity in captured representations, while a channel compression module distills task-relevant information. A weight sharing strategy is introduced among ART blocks to mitigate computational burden. A unified implementation is introduced to avoid the need to rebuild separate synthesis models for varying source-target modality configurations. Comprehensive demonstrations are performed for synthesizing missing sequences in multi-contrast MRI, and CT images from MRI. Our results indicate superiority of ResViT against competing CNN- and transformer-based methods in terms of qualitative observations and quantitative metrics.
Close relatives (of Feedback Vertex Set), revisited

At IPEC 2020, Bergougnoux, Bonnet, Brettell, and Kwon showed that a number of problems related to the classic Feedback Vertex Set (FVS) problem do not admit a $2^{o(k \log k)} \cdot n^{\mathcal{O}(1)}$-time algorithm on graphs of treewidth at most $k$, assuming the Exponential Time Hypothesis. This contrasts with the $3^{k} \cdot k^{\mathcal{O}(1)} \cdot n$-time algorithm for FVS using the Cut&Count technique. During their live talk at IPEC 2020, Bergougnoux et al.~posed a number of open questions, which we answer in this work. - Subset Even Cycle Transversal, Subset Odd Cycle Transversal, Subset Feedback Vertex Set can be solved in time $2^{\mathcal{O}(k \log k)} \cdot n$ in graphs of treewidth at most $k$. This matches a lower bound for Even Cycle Transversal of Bergougnoux et al.~and improves the polynomial factor in some of their upper bounds. - Subset Feedback Vertex Set and Node Multiway Cut can be solved in time $2^{\mathcal{O}(k \log k)} \cdot n$, if the input graph is given as a clique-width expression of size $n$ and width $k$. - Odd Cycle Transversal can be solved in time $4^k \cdot k^{\mathcal{O}(1)} \cdot n$ if the input graph is given as a clique-width expression of size $n$ and width $k$. Furthermore, the existence of a constant $\varepsilon>0$ and an algorithm performing this task in time $(4-\varepsilon)^k \cdot n^{\mathcal{O}(1)}$ would contradict the Strong Exponential Time Hypothesis.
Critical drag as a mechanism for resistivity

A quantum many-body system with a conserved electric charge can have a DC resistivity that is either exactly zero (implying it supports dissipationless current) or nonzero. Exactly zero resistivity is related to conservation laws that prevent the current from degrading. In this paper, we carefully examine the situations in which such a circumstance can occur. We find that exactly zero resistivity requires either continuous translation symmetry, or an internal symmetry that has a certain kind of "mixed anomaly" with the electric charge. (The symmetry could be a generalized global symmetry associated with the emergence of unbreakable loop or higher dimensional excitations.) However, even if one of these is satisfied, we show that there is still a mechanism to get nonzero resistivity, through critical fluctuations that drive the susceptibility of the conserved quantity to infinity; we call this mechanism "critical drag". Critical drag is thus a mechanism for resistivity that, unlike conventional mechanisms, is unrelated to broken symmetries. We furthermore argue that an emergent symmetry that has the appropriate mixed anomaly with electric charge is in fact an inevitable consequence of compressibility in systems with lattice translation symmetry. Critical drag therefore seems to be the only way (other than through irrelevant perturbations breaking the emergent symmetry, that disappear at the renormalization group fixed point) to get nonzero resistivity in such systems. Finally, we present a very simple and concrete model -- the "Quantum Lifshitz Model" -- that illustrates the critical drag mechanism as well as the other considerations of the paper.
Trace anomaly, Perelman’s functionals and the cosmological constant

The trace anomaly and the cosmological constant problem are two typical breakdowns when applying the quantum principle to a general covariant or gravitational system. A quantum theory of spacetime reference frame is proposed and reviewed. We study the theory by functional method, and show that the trace anomaly of the theory is closely related to some of the Perelman’s functionals. The functionals may provide us possible links between the trace anomaly and the cosmological constant. We find that to cancel the trace anomaly at the lab’s scale up to very high energy, a cosmological constant is required which is consistent with observations. In the framework, an effective theory of gravity and possible observational effect are also discussed.
End‐periodic homeomorphisms and volumes of mapping tori

Given an irreducible, end‐periodic homeomorphism f:S→S$f: S \rightarrow S$ of a surface with finitely many ends, all accumulated by genus, the mapping torus, Mf$M_f$ , is the interior of a compact, irreducible, atoroidal 3‐manifold M¯f$\overline{M}_f$ with incompressible boundary. Our main result is an upper bound on the infimal hyperbolic volume of M¯f$\overline{M}_f$ in terms of the translation length of f$f$ on the pants graph of S$S$ . This builds on work of Brock and Agol in the finite‐type setting. We also construct a broad class of examples of irreducible, end‐periodic homeomorphisms and use them to show that our bound is asymptotically sharp.
Testing normality of spatially indexed functional data

We develop a test of normality for spatially indexed functions. The assumption of normality is common in spatial statistics, yet no significance tests, or other means of assessment, have been available for functional data. This article aims at filling this gap in the case of functional observations on a spatial grid. Our test compares the moments of the spatial (frequency domain) principal component scores to those of a suitable Gaussian distribution. Critical values can be readily obtained from a chi‐squared distribution. We provide rigorous theoretical justification for a broad class of weakly stationary functional random fields. We perform simulation studies to assess the power of the test against various alternatives. An application to surface incoming shortwave radiation illustrates the practical value of this procedure.
The MultiBERTs: BERT Reproductions for Robustness Analysis

Experiments with pre-trained models such as BERT are often based on a single checkpoint. While the conclusions drawn apply to the artifact tested in the experiment (i.e., the particular instance of the model), it is not always clear whether they hold for the more general procedure which includes the architecture, training data, initialization scheme, and loss function. Recent work has shown that repeating the pre-training process can lead to substantially different performance, suggesting that an alternate strategy is needed to make principled statements about procedures. To enable researchers to draw more robust conclusions, we introduce the MultiBERTs, a set of 25 BERT-Base checkpoints, trained with similar hyper-parameters as the original BERT model but differing in random weight initialization and shuffling of training data. We also define the Multi-Bootstrap, a non-parametric bootstrap method for statistical inference designed for settings where there are multiple pre-trained models and limited test data. To illustrate our approach, we present a case study of gender bias in coreference resolution, in which the Multi-Bootstrap lets us measure effects that may not be detected with a single checkpoint. We release our models and statistical library along with an additional set of 140 intermediate checkpoints captured during pre-training to facilitate research on learning dynamics.
Constraining Mornings and Evenings on Distant Worlds: A new Semianalytical Approach and Prospects with Transmission Spectroscopy

The technique of transmission spectroscopy—the variation of a planetary radius with wavelength due to opacity sources in the planet’s terminator region—has been one of the most successful in the characterization of exoplanet atmospheres to date, providing key insights into the composition and structure of these distant worlds. A common assumption made when using this technique, however, is that the variations are the same in the entire terminator region. In reality, the morning and evening terminators might have distinct temperature, pressure, and thus compositional profiles due to the inherent 3D nature of the planet, which would, in turn, give rise to different spectra on each side of it. Constraining those might be fundamental for our understanding of not only the weather patterns in these distant worlds but also the planetary formation signatures that might only be possible to extract once these features are well understood. Motivated by this physical picture, in this work, we perform a detailed study of the observational prospects of detecting this effect. We present an open-source semianalytical framework with which this information can be extracted directly from transit light curves and perform a detailed study of the prospects of detecting the effect with current missions, such as TESS, and upcoming ones, such as JWST. Our results show that these missions show great promise for the detection of this effect. Transmission spectroscopy studies with JWST, in particular, could provide spectra of each of the limbs, allowing us to convey 3D information previously accessible only via phase curves.
Early Risk Detection of Pathological Gambling, Self-Harm and Depression Using BERT

Early risk detection of mental illnesses has a massive positive impact upon the well-being of people. The eRisk workshop has been at the forefront of enabling interdisciplinary research in developing computational methods to automatically estimate early risk factors for mental issues such as depression, self-harm, anorexia and pathological gambling. In this paper, we present the contributions of the BLUE team in the 2021 edition of the workshop, in which we tackle the problems of early detection of gambling addiction, self-harm and estimating depression severity from social media posts. We employ pre-trained BERT transformers and data crawled automatically from mental health subreddits and obtain reasonable results on all three tasks.