Miracle-dz/UniFAR-FTUs huggingface dataset:
OTSeq2Set: An Optimal Transport Enhanced Sequence-to-Set Model for Extreme Multi-label Text Classification

Extreme multi-label text classification (XMTC) is the task of finding the most relevant subset labels from an extremely large-scale label collection. Recently, some deep learning models have achieved state-of-the-art results in XMTC tasks. These models commonly predict scores for all labels by a fully connected layer as the last layer of the model. However, such models can’t predict a relatively complete and variable-length label subset for each document, because they select positive labels relevant to the document by a fixed threshold or take top k labels in descending order of scores. A less popular type of deep learning models called sequence-to-sequence (Seq2Seq) focus on predicting variable-length positive labels in sequence style. However, the labels in XMTC tasks are essentially an unordered set rather than an ordered sequence, the default order of labels restrains Seq2Seq models in training. To address this limitation in Seq2Seq, we propose an autoregressive sequence-to-set model for XMTC tasks named OTSeq2Set. Our model generates predictions in student-forcing scheme and is trained by a loss function based on bipartite matching which enables permutation-invariance. Meanwhile, we use the optimal transport distance as a measurement to force the model to focus on the closest labels in semantic label space. Experiments show that OTSeq2Set outperforms other competitive baselines on 4 benchmark datasets. Especially, on the Wikipedia dataset with 31k labels, it outperforms the state-of-the-art Seq2Seq method by 16.34% in micro-F1 score. The code is available at https://github.com/caojie54/OTSeq2Set.
On the Curious Case of $\ell_2$ norm of Sense Embeddings

We show that the $\ell_2$ norm of a static sense embedding encodes information related to the frequency of that sense in the training corpus used to learn the sense embeddings. This finding can be seen as an extension of a previously known relationship for word embeddings to sense embeddings. Our experimental results show that, in spite of its simplicity, the $\ell_2$ norm of sense embeddings is a surprisingly effective feature for several word sense related tasks such as (a) most frequent sense prediction, (b) Word-in-Context (WiC), and (c) Word Sense Disambiguation (WSD). In particular, by simply including the $\ell_2$ norm of a sense embedding as a feature in a classifier, we show that we can improve WiC and WSD methods that use static sense embeddings.
ProVe: A Pipeline for Automated Provenance Verification of Knowledge Graphs against Textual Sources

Knowledge Graphs are repositories of information that gather data from a multitude of domains and sources in the form of semantic triples, serving as a source of structured data for various crucial applications in the modern web landscape, from Wikipedia infoboxes to search engines. Such graphs mainly serve as secondary sources of information and depend on well-documented and verifiable provenance to ensure their trustworthiness and usability. However, their ability to systematically assess and assure the quality of this provenance, most crucially whether it properly supports the graph’s information, relies mainly on manual processes that do not scale with size. ProVe aims at remedying this, consisting of a pipelined approach that automatically verifies whether a Knowledge Graph triple is supported by text extracted from its documented provenance. ProVe is intended to assist information curators and consists of four main steps involving rule-based methods and machine learning models: text extraction, triple verbalisation, sentence selection, and claim verification. ProVe is evaluated on a Wikidata dataset, achieving promising results overall and excellent performance on the binary classification task of detecting support from provenance, with 87.5 % accuracy and 82.9 % F1-macro on text-rich sources. The evaluation data and scripts used in this paper are available in GitHub and Figshare.
Hybrid Inlining: A Compositional and Context Sensitive Static Analysis Framework

Context sensitivity is essential for achieving the precision in inter-procedural static analysis. To be (fully) context sensitive, top-down analysis needs to fully inline all statements of the callees at each callsite, leading to statement explosion. Compositional analysis, which inlines summaries of the callees, scales up but often loses precision, as it is not strictly context sensitive. We propose a compositional and strictly context sensitive framework for static analysis. This framework is based on one key observation: a compositional static analysis often loses precision only on some critical statements that need to be analyzed context sensitively. Our approach hybridly inlines the critical statements and the summaries of non-critical statements of each callee, thus avoiding the re-analysis of non-critical ones. In addition, our analysis lazily summarizes the critical statements, by stopping propagating the critical statements once the calling context accumulated is adequate. Hybrid Inlining can be as precise as context sensitive top-down analysis. We have designed and implemented a pointer analysis based on this framework. It can analyze large Java programs from the Dacapo benchmark suite and industry in minutes. In our evaluation, compared to context insensitive analysis, Hybrid Inlining just brings 65% and 1% additional time overhead on Dacapo and industrial applications respectively.
A Noise-aware Transpiler for Optimal Amplitude Amplification

Amplitude ampliﬁcation provides a quadratic speed-up for an array of quantum algorithms when run on a quantum machine perfectly isolated from its environment. However, the advantage is substantially diminished as the NISQ-era quantum machines lack the large number of qubits necessary to provide error correction. Noise in the computation grows with the number of gate counts in the circuit with each iteration of amplitude ampliﬁcation. After a certain number of ampliﬁcations, the loss in accuracy from the gate noise starts to overshadow the gain in accuracy due to ampliﬁcation, forming an inﬂection point. Beyond this point, accuracy continues to deteriorate until the machine reaches a maximally mixed state where the result is uniformly random. Hence, quantum transpilers should take the noise parameters of the underlying quantum machine into consideration such that the circuit can be optimized to attain the maximal accuracy possible for that machine. In this work, we propose an extension to the transpiler that predicts the accuracy of the result at every ampliﬁcation with high ﬁdelity by applying pure Bayesian analysis to individual gate noise rates. Using this information, it ﬁnds the inﬂection point and optimizes the circuit by halting ampliﬁcation at that point. The prediction is made without needing to execute the circuit either on a quantum simulator or an actual quantum machine
Tracking-based distributed equilibrium seeking for aggregative games

We propose fully-distributed algorithms for Nash equilibrium seeking in aggregative games over networks. We first consider the case where local constraints are present and we design an algorithm combining, for each agent, (i) the projected pseudo-gradient descent and (ii) a tracking mechanism to locally reconstruct the aggregative variable. To handle coupling constraints arising in generalized settings, we propose another distributed algorithm based on (i) a recently emerged augmented primal-dual scheme and (ii) two tracking mechanisms to reconstruct, for each agent, both the aggregative variable and the coupling constraint satisfaction. Leveraging tools from singular perturbations analysis, we prove linear convergence to the Nash equilibrium for both schemes. Finally, we run extensive numerical simulations to confirm the effectiveness of our methods and compare them with state-of-the-art distributed equilibrium-seeking algorithms.
ANACONDA: An Improved Dynamic Regret Algorithm for Adaptive Non-Stationary Dueling Bandits

We study the problem of non-stationary dueling bandits and provide the first adaptive dynamic regret algorithm for this problem. The only two existing attempts in this line of work fall short across multiple dimensions, including pessimistic measures of non-stationary complexity and non-adaptive parameter tuning that requires knowledge of the number of preference changes. We develop an elimination-based rescheduling algorithm to overcome these shortcomings and show a near-optimal $\tilde{O}(\sqrt{S^{\texttt{CW}} T})$ dynamic regret bound, where $S^{\texttt{CW}}$ is the number of times the Condorcet winner changes in $T$ rounds. This yields the first near-optimal dynamic regret algorithm for unknown $S^{\texttt{CW}}$. We further study other related notions of non-stationarity for which we also prove near-optimal dynamic regret guarantees under additional assumptions on the underlying preference model.
A novel approach to correcting Te-based mass-metallicity relations

Deriving oxygen abundances from the electron temperature (hereafter the Te-method) is the gold-standard for extragalactic metallicity studies. However, unresolved temperature fluctuations within individual H ii regions and across different H ii regions throughout a galaxy can bias metallicity estimates low, with a magnitude that depends on the underlying and typically unknown temperature distribution. Using a toy model, we confirm that computing Te-based metallicities using the temperature derived from the [O iii] λ4363/λ5007 or [O ii] λλ7320,7330 / [O ii] λλ3727 ratio (‘ratio temperature’; Tratio) results in an underprediction of metallicity when temperature fluctuations are present. In contrast, using the unobservable ‘line temperatures’ (Tline) that provide the mean electron and ion density-weighted emissivity yield an accurate metallicity estimate. To correct this bias in low-mass galaxies, we demonstrate an example calibration of a relation between Tratio and Tline based on a high-resolution (4.5 pc) RAMSES-RTZ simulation of a dwarf galaxy that self-consistently models the formation of multiple H ii regions and ion temperature distribution in a galactic context. Applying this correction to the low-mass end of the mass-metallicity relation shifts its normalization up by 0.18 dex on average and flattens its slope from 0.87 to 0.58, highlighting the need for future studies to account for, and correct, this bias.
Super-Resolution Based Patch-Free 3D Medical Image Segmentation with Self-Supervised Guidance

—High resolution (HR) 3D medical image segmentation plays an important role in clinical diagnoses. However, HR images are difﬁcult to be directly processed by mainstream graphical cards due to limited video memory. Therefore, most existing 3D medical image segmentation methods use patch-based models, which ignores global context information that is useful in accurate segmentation and has low inference efﬁciency. To address these problems, we propose a super-resolution (SR) guided patch-free 3D medical image segmentation framework that can realize HR segmentation with global information of low-resolution (LR) input. The framework contains two tasks: semantic segmentation (main task) and super resolution (auxiliary task). To balance the information loss with the LR input, we introduce a Self-Supervised Guidance Module (SGM), which employs a selective search method to crop a HR patch from the original image as restoration guidance. Multi-scale convolutional layers are used to mitigate the scale-inconsistency between the HR guidance features and the LR features. Moreover, we propose a Task-Fusion Module (TFM) to exploit the inter connections between segmentation and SR task. This module can also be used for Test Phase Fine-tuning (TPF), leading to a better model generalization ability. When predicting, only the main segmentation task is
Text-to-speech synthesis from dark data with evaluation-in-the-loop data selection

This paper proposes a method for selecting training data for textto-speech (TTS) synthesis from dark data. TTS models are typically trained on high-quality speech corpora that cost much time and money for data collection, which makes it very challenging to increase speaker variation. In contrast, there is a large amount of data whose availability is unknown (a.k.a, “dark data”), such as YouTube videos. To utilize data other than TTS corpora, previous studies have selected speech data from the corpora on the basis of acoustic quality. However, considering that TTS models robust to data noise have been proposed, we should select data on the basis of its importance as training data to the given TTS model, not the quality of speech itself. Our method with a loop of training and evaluation selects training data on the basis of the automatically predicted quality of synthetic speech of a given TTS model. Results of evaluations using YouTube data reveal that our method outperforms the conventional acoustic-quality-based method.
End-to-End Multimodal Representation Learning for Video Dialog

Video-based dialog task is a challenging multimodal learning task that has received increasing attention over the past few years with state-of-the-art obtaining new performance records. This progress is largely powered by the adaptation of the more powerful transformer-based language encoders. Despite this progress, existing approaches do not effectively utilize visual features to help solve tasks. Recent studies show that state-of-the-art models are biased toward textual information rather than visual cues. In order to better leverage the available visual information, this study proposes a new framework that combines 3D-CNN network and transformer-based networks into a single visual encoder to extract more robust semantic representations from videos. The visual encoder is jointly trained end-to-end with other input modalities such as text and audio. Experiments on the AVSD task show significant improvement over baselines in both generative and retrieval tasks.
Hierarchical Federated Learning With Momentum Acceleration in Multi-Tier Networks

In this article, we propose Hierarchical Federated Learning with Momentum Acceleration (HierMo), a three-tier worker-edge-cloud federated learning algorithm that applies momentum for training acceleration. Momentum is calculated and aggregated in the three tiers. We provide convergence analysis for HierMo, showing a convergence rate of <inline-formula><tex-math notation="LaTeX">$\mathcal {O}(\frac{1}{T})$</tex-math><alternatives><mml:math><mml:mrow><mml:mi mathvariant="script">O</mml:mi><mml:mo>(</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mi>T</mml:mi></mml:mfrac><mml:mo>)</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="bao-ieq1-3294688.gif"/></alternatives></inline-formula>. In the analysis, we develop a new approach to characterize model aggregation, momentum aggregation, and their interactions. Based on this result, we prove that HierMo achieves a tighter convergence upper bound compared with HierFAVG without momentum. We also propose HierOPT, which optimizes the aggregation periods (worker-edge and edge-cloud aggregation periods) to minimize the loss given a limited training time. By conducting the experiment, we verify that HierMo outperforms existing mainstream benchmarks under a wide range of settings. In addition, HierOPT can achieve a near-optimal performance when we test HierMo under different aggregation periods.
Analysis of the structural characteristics and optoelectronic properties of CaTiO3 as a non-toxic raw material for solar cells: a DFT study

: - Structural and optoelectronic properties of α, β, γ phases of calcium titanate are studied with the implementation of first-principles quantum-chemical calculations in the framework of DFT. When optimizing the geometry, the GGA approximation was used. The relaxed lattice parameters obtained by us are identical with the experimental analogs. It has been established that the most stable phase of calcium titanate is the orthorhombic syngony, which corresponds to the results of experimental measurements. It is shown that as the transition from the α-phase to the γ-phase proceeds, the lattice constant parameters and the interatomic distance in these systems increase. The optoelectronic properties of these materials have been studied using the Wien2k code. The high-precision TB-mBJ approximation was used to calculate the exchange-correlation effects. An analysis of the electronic properties of these materials showed that all the studied phases of calcium titanate belong to the class of wide-gap semiconductors. The calculated band gaps for the cubic, tetragonal, and orthorhombic CaTiO 3 systems are 2.83, 3.07, and 3.26 eV, respectively. According to the analysis of DOS-plots, it was found that the tetragonal phase of calcium titanate is characterized by the highest density of states. Calculations of the optical constants of the systems under study showed that the CaTiO 3 cubic system is characterized by an increased absorption capacity and a relatively high photoconductivity. However, for the other two phases of calcium titanate, the calculations gave identical patterns, i.e., the absorption and optical conductivity spectra of the tetragonal and orthorhombic CaTiO 3 systems practically coincide.
Quantitative elemental imaging in eukaryotic algae

Abstract All organisms, fundamentally, are made from the same raw material, namely the elements of the periodic table. Biochemical diversity is achieved by how these elements are utilized, for what purpose, and in which physical location. Determining elemental distributions, especially those of trace elements that facilitate metabolism as cofactors in the active centers of essential enzymes, can determine the state of metabolism, the nutritional status, or the developmental stage of an organism. Photosynthetic eukaryotes, especially algae, are excellent subjects for quantitative analysis of elemental distribution. These microbes utilize unique metabolic pathways that require various trace nutrients at their core to enable their operation. Photosynthetic microbes also have important environmental roles as primary producers in habitats with limited nutrient supplies or toxin contaminations. Accordingly, photosynthetic eukaryotes are of great interest for biotechnological exploitation, carbon sequestration, and bioremediation, with many of the applications involving various trace elements and consequently affecting their quota and intracellular distribution. A number of diverse applications were developed for elemental imaging, allowing subcellular resolution, with X-ray fluorescence microscopy (XFM, XRF) being at the forefront, enabling quantitative descriptions of intact cells in a non-destructive method. This Tutorial Review summarizes the workflow of a quantitative, single-cell elemental distribution analysis of a eukaryotic alga using XFM.
Deep Metric Learning with Adaptive Margin and Adaptive Scale for Acoustic Word Discrimination

Many recent loss functions in deep metric learning are expressed with logarithmic and exponential forms, and they involve margin and scale as essential hyper-parameters. Since each data class has an intrinsic characteristic, several previous works have tried to learn embedding space close to the real distribution by introducing adaptive margins. However, there was no work on adaptive scales at all. We argue that both margin and scale should be adaptively adjustable during the training. In this paper, we propose a method called Adaptive Margin and Scale (AdaMS), where hyper-parameters of margin and scale are replaced with learnable parameters of adaptive margins and adaptive scales for each class. Our method is evaluated on Wall Street Journal dataset, and we achieve outperforming results for word discrimination tasks.
On multiscale quasi-interpolation of scattered scalar- and manifold-valued functions

We address the problem of approximating an unknown function from its discrete samples given at arbitrarily scattered sites. This problem is essential in numerical sciences, where modern applications also highlight the need for a solution to the case of functions with manifold values. In this paper, we introduce and analyze a combination of kernel-based quasi-interpolation and multiscale approximations for both scalar- and manifold-valued functions. While quasi-interpolation provides a powerful tool for approximation problems if the data is defined on infinite grids, the situation is more complicated when it comes to scattered data. Here, higher-order quasi-interpolation schemes either require derivative information or become numerically unstable. Hence, this paper principally studies the improvement achieved by combining quasi-interpolation with a multiscale technique. The main contributions of this paper are as follows. First, we introduce the multiscale quasi-interpolation technique for scalar-valued functions. Second, we show how this technique can be carried over using moving least-squares operators to the manifold-valued setting. Third, we give a mathematical proof that converging quasi-interpolation will also lead to converging multiscale quasi-interpolation. Fourth, we provide ample numerical evidence that multiscale quasi-interpolation has superior convergence to quasi-interpolation. In addition, we will provide examples showing that the multiscale quasi-interpolation approach offers a powerful tool for many data analysis tasks, such as denoising and anomaly detection. It is especially attractive for cases of massive data points and high dimensionality.
How Precise are Performance Estimates for Typical Medical Image Segmentation Tasks?

An important issue in medical image processing is to be able to estimate not only the performances of algorithms but also the precision of the estimation of these performances. Reporting precision typically amounts to reporting standard-error of the mean (SEM) or equivalently confidence intervals. However, this is rarely done in medical image segmentation studies. In this paper, we aim to estimate what is the typical confidence that can be expected in such studies. To that end, we first perform experiments for Dice metric estimation using a standard deep learning model (U-net) and a classical task from the Medical Segmentation Decathlon. We extensively study precision estimation using both Gaussian assumption and bootstrapping (which does not require any assumption on the distribution). We then perform simulations for other test set sizes and performance spreads. Overall, our work shows that small test sets lead to wide confidence intervals (e.g. ~8 points of Dice for 20 samples with σ ≃ 10).
A Data-constrained Magnetohydrodynamic Simulation of the X1.0 Solar Flare of 2021 October 28

The solar active region NOAA 12887 produced a strong X1.0 flare on 2021 October 28, which exhibits X-shaped flare ribbons and a circle-shaped erupting filament. To understand the eruption process with these characteristics, we conducted a data-constrained magnetohydrodynamics simulation using a nonlinear force-free field of the active region about an hour before the flare as the initial condition. Our simulation reproduces the filament eruption observed in the Hα images of GONG and the 304 Å images of SDO/AIA, and suggests that two mechanisms can possibly contribute to the magnetic eruption. One is the torus instability of the preexisting magnetic flux rope (MFR) and the other is upward pushing by magnetic loops newly formed below the MFR via continuous magnetic reconnection between two sheared magnetic arcades. The presence of this reconnection is evidenced by the SDO/AIA observations of the 1600 Å brightening in the footpoints of the sheared arcades at the flare onset. To clarify which process is more essential for the eruption, we performed an experimental simulation in which the reconnection between the sheared field lines is suppressed. In this case too, the MFR could erupt, but at a much reduced rising speed. We interpret this result as indicating that the eruption is not only driven by the torus instability, but additionally accelerated by newly formed and rising magnetic loops under continuous reconnection.