Miracle-dz/UniFAR-FTUs huggingface dataset:
Deep Semi-supervised Learning for Time Series Classification

While deep semi-supervised learning has gained much attention in computer vision, limited research exists on its applicability in the time series domain. In this work, we investigate the transferability of state-of-the-art deep semi-supervised models from image to time series classification. We discuss the necessary model adaptations, in particular an appropriate model backbone architecture and the use of tailored data augmentation strategies. Based on these adaptations, we explore the potential of deep semi-supervised learning in the context of time series classification by evaluating our methods on large public time series classification problems with varying amounts of labeled samples. We perform extensive comparisons under a decidedly realistic and appropriate evaluation scheme with a unified reimplementation of all algorithms considered, which is yet lacking in the field. We find that these transferred semi-supervised models show significant performance gains over strong supervised, semi-supervised and self-supervised alternatives, especially for scenarios with very few labeled samples.
Nyströmformer: A Nyström-Based Algorithm for Approximating Self-Attention

Transformers have emerged as a powerful tool for a broad range of natural language processing tasks. A key component that drives the impressive performance of Transformers is the self-attention mechanism that encodes the influence or dependence of other tokens on each specific token. While beneficial, the quadratic complexity of self-attention on the input sequence length has limited its application to longer sequences - a topic being actively studied in the community. To address this limitation, we propose Nyströmformer - a model that exhibits favorable scalability as a function of sequence length. Our idea is based on adapting the Nyström method to approximate standard self-attention with O(n) complexity. The scalability of Nyströmformer enables application to longer sequences with thousands of tokens. We perform evaluations on multiple downstream tasks on the GLUE benchmark and IMDB reviews with standard sequence length, and find that our Nyströmformer performs comparably, or in a few cases, even slightly better, than standard self-attention. On longer sequence tasks in the Long Range Arena (LRA) benchmark, Nyströmformer performs favorably relative to other efficient self-attention methods. Our code is available at https://github.com/mlpen/Nystromformer.
MILD COMBUSTION FOR FUEL-NOx REDUCTION

One source of pollution in the combustion of fossil fuels is the formation of nitrogen oxides. Their production is strictly bound to the presence of molecular nitrogen in the combustion air and to the so-called fuel nitrogen, that is, the N chemically bound to the fuel. Thermal NOx formation, which is the most important source of nitrogen oxides in high-temperature combustion processes, can be limited and controlled by the relatively new “mild” technology, also called “flameless” combustion. However, if a fuel containing some nitrogen is used, other ways of NOx production become important. The aim of this study was to investigate the possibility of reducing fuel-NOx emissions using a laboratory-scale burner for mild combustion. Two selective noncatalytic reduction technologies, namely a reburning-like and thermal DeNOx-like have been coupled with mild combustion to further reduce NOx formation and the experimental results have been discussed using detailed kinetic calculations.
Concentration of Non-Isotropic Random Tensors with Applications to Learning and Empirical Risk Minimization

Dimension is an inherent bottleneck to some modern learning tasks, where optimization methods suffer from the size of the data. In this paper, we study non-isotropic distributions of data and develop tools that aim at reducing these dimensional costs by a dependency on an effective dimension rather than the ambient one. Based on non-asymptotic estimates of the metric entropy of ellipsoids -- that prove to generalize to infinite dimensions -- and on a chaining argument, our uniform concentration bounds involve an effective dimension instead of the global dimension, improving over existing results. We show the importance of taking advantage of non-isotropic properties in learning problems with the following applications: i) we improve state-of-the-art results in statistical preconditioning for communication-efficient distributed optimization, ii) we introduce a non-isotropic randomized smoothing for non-smooth optimization. Both applications cover a class of functions that encompasses empirical risk minization (ERM) for linear models.
Decomposition of residues and loss of the δ-endotoxin from transgenic (Bt) corn (Zea mays L.) in soil

Corn and other crops genetically modified to express the insecticidal δ-endotoxin from Bacillus thuringiensis (Bt) are grown widely across north America. Studies have shown that the δ -endotoxin can be stabilised on soil colloids where its activity is retained, but reports of direct ecological effects of the δ-endotoxin on soil processes are limited. We have determined the concentrations of the δ-endotoxin in organic residues fro m Bt-corn plants at increasing stages of ageing and decay, and the subsequent decomposition in soil of these residues and the δ-endotoxin in them. The δ-endotoxin concentrations declined from 6.8 μg g-1 in the fresh plant material, to 0.82 μg g-1 in the post-harvest residues collected in the fall, and to 0.026 μg g-1 in the residues collected from soil surface the following spring. The concentration of δ -endotoxin in buried residues collected in the spring was not significantly different from zero. When incubated in soil in the laboratory over 84 d, the δ-endotoxin decomposed mo.
Revisiting Newly Large Magellanic Cloud Age-gap Star Clusters

Recently, a noticeable number of new star clusters was identified in the outskirts of the Large Magellanic Cloud (LMC) populating the so-called star-cluster age gap, a space of time (∼4–12 Gyr) where the only known star cluster is up-to-date ESO 121-SC 03. We used Survey of the Magellanic Stellar History DR2 data sets, as well as those employed to identify these star-cluster candidates, to produce relatively deep color–magnitude diagrams (CMDs) of 17 out of 20 discovered age-gap star clusters with the aim of investigating them in detail. Our analysis relies on a thorough CMD cleaning procedure of the field-star contamination, which presents variations in its stellar density and astrophysical properties, such as luminosity and effective temperature, around the star-cluster fields. We built star-cluster CMDs from stars with membership probabilities assigned from the cleaning procedure. These CMDs and their respective spatial distribution maps favor the existence of LMC star field density fluctuations rather than age-gap star clusters, although a definitive assessment on them will be possible from further deeper photometry.
On the noncollapsedness of positively curved Type I ancient Ricci flows

In this paper, we study complete Type I ancient Ricci flows with positive sectional curvature. Our main results are as follows: in the complete and noncompact case, all such ancient solutions must be noncollapsed on all scales; in the closed case, if the dimension is even, then all such ancient solutions must be noncollapsed on all scales. This furthermore gives a complete classification for three‐dimensional noncompact Type I ancient solutions without assuming the noncollapsing condition.
Polynomial Linear System Solving with Random Errors: New Bounds and Early Termination Technique

This paper deals with the polynomial linear system solving with errors (PLSwE) problem. More specifically, we solve linear systems with univariate polynomial coefficients via an evaluation-interpolation technique assuming that errors can occur before the interpolation step. In this framework, the number of evaluations needed to recover the solution depends on the parameters of the linear system (degrees, size) and on the number of errors. Our work is part of a series of papers about PLSwE aiming to reduce this number of evaluations, which is crucial since it affects the complexity. We proved in [Guerrini et al., Proc. ISIT'19] that if errors are randomly distributed, the bound on the number of evaluations can be lowered with respect to the error rate. In this paper, following the approach of [Kaltofen et al., Proc. ISSAC'17], we improve the results of [Guerrini et al., Proc. ISIT'19] in two directions. First, we propose a new bound on the number of evaluations, lowering the dependency on the parameters of the linear system, based on the work of [Cabay, Proc. SYMSAC'71]. Second, we introduce an early termination strategy in order to handle the unnecessary increase of the number of evaluations due to the overestimation of the output degrees and of the number of errors.
Escaping Stochastic Traps with Aleatoric Mapping Agents

Exploration in environments with sparse rewards is difficult for artificial agents. Curiosity driven learning -- using feed-forward prediction errors as intrinsic rewards -- has achieved some success in these scenarios, but fails when faced with action-dependent noise sources. We present aleatoric mapping agents (AMAs), a neuroscience inspired solution modeled on the cholinergic system of the mammalian brain. AMAs aim to explicitly ascertain which dynamics of the environment are unpredictable, regardless of whether those dynamics are induced by the actions of the agent. This is achieved by generating separate forward predictions for the mean and variance of future states and reducing intrinsic rewards for those transitions with high aleatoric variance. We show AMAs are able to effectively circumvent action-dependent stochastic traps that immobilise conventional curiosity driven agents. The code for all experiments presented in this paper is open sourced: http://github.com/self-supervisor/Escaping-Stochastic-Traps-With-Aleatoric-Mapping-Agents.
Online Limited Memory Neural-Linear Bandits with Likelihood Matching

We study neural-linear bandits for solving problems where {\em both} exploration and representation learning play an important role. Neural-linear bandits harnesses the representation power of Deep Neural Networks (DNNs) and combines it with efficient exploration mechanisms by leveraging uncertainty estimation of the model, designed for linear contextual bandits on top of the last hidden layer. In order to mitigate the problem of representation change during the process, new uncertainty estimations are computed using stored data from an unlimited buffer. Nevertheless, when the amount of stored data is limited, a phenomenon called catastrophic forgetting emerges. To alleviate this, we propose a likelihood matching algorithm that is resilient to catastrophic forgetting and is completely online. We applied our algorithm, Limited Memory Neural-Linear with Likelihood Matching (NeuralLinear-LiM2) on a variety of datasets and observed that our algorithm achieves comparable performance to the unlimited memory approach while exhibits resilience to catastrophic forgetting.
Quasi-periodic Particle Acceleration in a Solar Flare

A common feature of electromagnetic emission from solar flares is the presence of intensity pulsations that vary as a function of time. Known as quasi-periodic pulsations (QPPs), these variations in flux appear to include periodic components and characteristic timescales. Here, we analyze a GOES M3.7 class flare exhibiting pronounced QPPs across a broad band of wavelengths using imaging and time series analysis. We identify QPPs in the time series of X-ray, low-frequency radio, and extreme ultraviolet (EUV) wavelengths using wavelet analysis, and localize the region of the flare site from which the QPPs originate via X-ray and EUV imaging. It was found that the pulsations within the 171 Å, 1600 Å, soft X-ray, and hard X-ray light curves yielded similar periods of s, s, s, and s, respectively, indicating a common progenitor. The low-frequency radio emission at 2.5 MHz contained a longer period of ∼231 s. Imaging analysis indicates that the location of the X-ray and EUV pulsations originates from a hard X-ray footpoint linked to a system of nearby open magnetic field lines. Our results suggest that intermittent particle acceleration, likely due to “bursty” magnetic reconnection, is responsible for the QPPs. The precipitating electrons accelerated toward the chromosphere produce the X-ray and EUV pulsations, while the escaping electrons result in low-frequency radio pulses in the form of type III radio bursts. The modulation of the reconnection process, resulting in episodic particle acceleration, explains the presence of these QPPs across the entire spatial range of flaring emission.
Colorization Transformer

We present the Colorization Transformer, a novel approach for diverse high fidelity image colorization based on self-attention. Given a grayscale image, the colorization proceeds in three steps. We first use a conditional autoregressive transformer to produce a low resolution coarse coloring of the grayscale image. Our architecture adopts conditional transformer layers to effectively condition grayscale input. Two subsequent fully parallel networks upsample the coarse colored low resolution image into a finely colored high resolution image. Sampling from the Colorization Transformer produces diverse colorings whose fidelity outperforms the previous state-of-the-art on colorising ImageNet based on FID results and based on a human evaluation in a Mechanical Turk test. Remarkably, in more than 60% of cases human evaluators prefer the highest rated among three generated colorings over the ground truth. The code and pre-trained checkpoints for Colorization Transformer are publicly available at https://github.com/google-research/google-research/tree/master/coltran
PAC-Bayes Bounds for Meta-learning with Data-Dependent Prior

By leveraging experience from previous tasks, meta-learning algorithms can achieve effective fast adaptation ability when encountering new tasks. However it is unclear how the generalization property applies to new tasks. Probably approximately correct (PAC) Bayes bound theory provides a theoretical framework to analyze the generalization performance for meta-learning. We derive three novel generalisation error bounds for meta-learning based on PAC-Bayes relative entropy bound. Furthermore, using the empirical risk minimization (ERM) method, a PAC-Bayes bound for meta-learning with data-dependent prior is developed. Experiments illustrate that the proposed three PAC-Bayes bounds for meta-learning guarantee a competitive generalization performance guarantee, and the extended PAC-Bayes bound with data-dependent prior can achieve rapid convergence ability.
Learning Audio-Visual Correlations From Variational Cross-Modal Generation

People can easily imagine the potential sound while seeing an event. This natural synchronization between audio and visual signals reveals their intrinsic correlations. To this end, we propose to learn the audio-visual correlations from the perspective of cross-modal generation in a self-supervised manner, the learned correlations can be then readily applied in multiple downstream tasks such as the audio-visual cross-modal localization and retrieval. We introduce a novel Variational AutoEncoder (VAE) framework that consists of Multiple encoders and a Shared decoder (MS-VAE) with an additional Wasserstein distance constraint to tackle the problem. Extensive experiments demonstrate that the optimized latent representation of the proposed MS-VAE can effectively learn the audio-visual correlations and can be readily applied in multiple audio-visual downstream tasks to achieve competitive performance even without any given label information during training.
Jointly Improving Language Understanding and Generation with Quality-Weighted Weak Supervision of Automatic Labeling

Neural natural language generation (NLG) and understanding (NLU) models are data-hungry and require massive amounts of annotated data to be competitive. Recent frameworks address this bottleneck with generative models that synthesize weak labels at scale, where a small amount of training labels are expert-curated and the rest of the data is automatically annotated. We follow that approach, by automatically constructing a large-scale weakly-labeled data with a fine-tuned GPT-2, and employ a semi-supervised framework to jointly train the NLG and NLU models. The proposed framework adapts the parameter updates to the models according to the estimated label-quality. On both the E2E and Weather benchmarks, we show that this weakly supervised training paradigm is an effective approach under low resource scenarios with as little as 10 data instances, and outperforming benchmark systems on both datasets when 100% of the training data is used.
Quantum isotropy and the reduction of dynamics in Bianchi I

The authors previously introduced a diffeomorphism-invariant definition of a homogeneous and isotropic sector of loop quantum gravity (LQG), along with a program to embed loop quantum cosmology (LQC) into it. The present paper works out that program in detail for the simpler, but still physically non-trivial, case where the target of the embedding is the homogeneous, but not isotropic, Bianchi I model. The diffeomorphism-invariant conditions imposing homogeneity and isotropy in the full theory reduce to conditions imposing isotropy on an already homogeneous Bianchi I spacetime. The reduced conditions are invariant under the residual diffeomorphisms still allowed after gauge fixing the Bianchi I model. We show that there is a unique embedding of the quantum isotropic model into the homogeneous quantum Bianchi I model that (a) is covariant with respect to the actions of such residual diffeomorphisms, and (b) intertwines both the (signed) volume operator and at least one directional Hubble rate. That embedding also intertwines all other operators of interest in the respective loop quantum cosmological models, including their Hamiltonian constraints. It thus establishes a precise equivalence between dynamics in the isotropic sector of the Bianchi I model and the quantized isotropic model, and not just their kinematics. We also discuss the adjoint relationship between the embedding map defined here and a projection map previously defined by Ashtekar and Wilson-Ewing. Finally, we highlight certain features that simplify this reduced embedding problem, but which may not have direct analogues in the embedding of homogeneous and isotropic LQC into full LQG.
Two-Step Image Dehazing with Intra-domain and Inter-domain Adaption

Recently, image dehazing task has achieved remarkable progress by convolutional neural network. However, those approaches mostly treat haze removal as a one-to-one problem and ignore the intra-domain gap. Therefore, haze distribution shift of the same scene images is not handled well. Also, dehazing models trained on the labeled synthetic datasets mostly suffer from performance degradation when tested on the unlabeled real datasets due to the inter-domain gap. Although some previous works apply translation network to bridge the synthetic domain and the real domain, the intra-domain gap still exists and affects the inter-domain adaption. In this work, we propose a novel Two-Step Dehazing Network (TSDN) to minimize the intra-domain gap and the inter-domain gap. First, we propose a multi-to-one dehazing network to eliminate the haze distribution shift of images within the synthetic domain. Then, we conduct an inter-domain adaption between the synthetic domain and the real domain based on the aligned synthetic features. Extensive experimental results demonstrate that our framework performs favorably against the state-of-the-art algorithms both on the synthetic datasets and the real datasets.
CHEMICAL CHARECTERIZATION OF VOLATILE OIL COMPONENTS OF CITRUS RETICULATA BY GC-MS ANALYSIS

Mandarin is a group name for a class of oranges with thin, loose peel, which have been dubbed "kid-glove" oranges. The loose-skinned mandarin orange (C. reticulate Blanco) is one of the most economically important and popular fruits in the world, constituting about 41% of the total citrus fruits produced in India. The main constituents of volatile oil is mixture of β-pinene, limonene and linalool α-terpineol, citropten, and an aldehyde tanageratin, citromitin, hesperidin and neohesperidin. The volatile constituents of volatile oil identified, % Area and RT of each constituent, are listed in table. Analysis of the isolate by GC and GC-MS resulted in the identification of fourteen components comprising 100 % of the total volatile constituents. With regard to flower parts volatiles, fourteen components (comprising 100%) were identified. The volatile oil was characterized by a high amount of monoterpene constituents (68.1%), monoterpenic hydrocarbons i.e. α-thujene(3.1%), α-pinene(4.2%), α-camphene(1.8%), β-pinene(10.8%) and Sabinene(0.9%),monoterpenic alcohol i.e. α-terpineol(0.8%), monoterpenic ketones.
Lead Migration from Ceramicware in Contact with Foodstuff: Effect of Glaze, Temperature, pH and Food Simulant

Lead (Pb) release from non-conforming flatwares and artisanal hollowwares was measured by ICP-AES. Influence of acidic beverages (tomato juice, vinegar and lemon juice), food simulants (acetic acid, citric acid, and malic acid) at pH (2.37-5.00), contact time (t) (30 min-2 days), temperature (T) (20 °C-90 °C), and glaze were investigated. Pb release was lowest in tomato juice and highest in lemon juice (t = 24 h; T = 22 °C). In acetic acid 4% (v/v) Pb release was 14% less than in lemon juice. Variability between the flatwares used for experiments in lemon juice and acetic acid 4% (v/v) corresponds to 3.48% and 7.03% respectively. According to the set of experiments where three food simulants were compared, it appeared that acetic acid, citric acid, and malic acid have the same leaching capacity above pH 3, but at pH < 3, citric acid appears to be the strongest extractant. Moreover, the influence of the applied ceramic glaze played an important role in lead release, in certain cases doubling the extracted amount of Pb. It has also been demonstrated that migration kinetics in citric acid is stronger. At 90 °C, after 2 h, the amount of extracted lead is 18% more important than at 20 °C after 24 h in acetic acid 4% (v/v) which may question the capacity of consumer protection of ED 84/500/EEC.
Long- and intermediate-range interaction in three lowest sigma states of the HeH+ ion

Accurate variational energies have been calculated for three lowest sigma states of the HeH+ ion. This includes the ground state (5 ≤ R ≤ 9 a.u.) which dissociates into He + H+, as well as the A1Σ+ state (4 ≤ R ≤ 10) and the a3Σ+ state (3 ≤ R ≤ 10) which both dissociate into He+ + H. The variational results are compared with those obtained using a perturbation theory expansion.
Bleaching of cotton fabric with peracetic acid in the presence of different activators

Peracetic acid is an efficient oxidant and a good alternative to conventional chemicals for bleaching cotton. In order to establish optimal conditions for its use, giving a high degree of whiteness and good water absorption without damage to the fabric, the chemistry of its consumption and the efficiency of the bleaching process have been examined. The effect of temperature and initial pH was assessed, and in particular the influence of the alkali used to control the pH. The highest and most rapid consumption of peracetic acid was seen with magnesium carbonate, the slowest with sodium hydroxide. The bleaching effect was greatest with sodium carbonate and least with sodium hydroxide, although the differences were not large. In all cases damage to the fabric was minimal. The best conditions for bleaching with peracetic acid were found to be 60 °C and an initial pH 7, in the presence of either sodium carbonate or magnesium carbonate.