Miracle-dz/UniFAR-FTUs huggingface dataset:
Contrast-PLC: Contrastive Learning for Packet Loss Concealment

Packet loss concealment (PLC) is challenging in concealing missing contents both plausibly and naturally when there are only limited available context to use. Recently deep-learning based PLC algorithms have demonstrated their superiority over traditional counterparts; but their concealment ability is still mostly limited to a maximum of 120ms loss. Even with strong GAN-based generative models, it is still very challenging to predict long burst losses that could happen within/in-between phonemes. In this paper, we propose to use contrastive learning to learn a loss-robust semantic representation for PLC. A hybrid neural PLC architecture combining the semantic prediction and GAN-based generative model is designed to verify its effectiveness. Results on the blind test set of Interspeech2022 PLC Challenge show its superiority over commonly used UNet-style framework and the one without contrastive learning, especially for the longer burst loss at (120, 220]ms.
DCLP: Neural Architecture Predictor with Curriculum Contrastive Learning

Neural predictors have shown great potential in the evaluation process of neural architecture search (NAS). However, current predictor-based approaches overlook the fact that training a predictor necessitates a considerable number of trained neural networks as the labeled training set, which is costly to obtain. Therefore, the critical issue in utilizing predictors for NAS is to train a high-performance predictor using as few trained neural networks as possible. Although some methods attempt to address this problem through unsupervised learning, they often result in inaccurate predictions. We argue that the unsupervised tasks intended for the common graph data are too challenging for neural networks, causing unsupervised training to be susceptible to performance crashes in NAS. To address this issue, we propose a CurricuLum-guided Contrastive Learning framework for neural Predictor (DCLP). Our method simplifies the contrastive task by designing a novel curriculum to enhance the stability of unlabeled training data distribution during contrastive training. Specifically, we propose a scheduler that ranks the training data according to the contrastive difficulty of each data and then inputs them to the contrastive learner in order. This approach concentrates the training data distribution and makes contrastive training more efficient. By using our method, the contrastive learner incrementally learns feature representations via unsupervised data on a smooth learning curve, avoiding performance crashes that may occur with excessively variable training data distributions. We experimentally demonstrate that DCLP has high accuracy and efficiency compared with existing predictors, and shows promising potential to discover superior architectures in various search spaces when combined with search strategies. Our code is available at: https://github.com/Zhengsh123/DCLP.
Better Generative Replay for Continual Federated Learning

Federated learning is a technique that enables a centralized server to learn from distributed clients via communications without accessing the client local data. However, existing federated learning works mainly focus on a single task scenario with static data. In this paper, we introduce the problem of continual federated learning, where clients incrementally learn new tasks and history data cannot be stored due to certain reasons, such as limited storage and data retention policy. Generative replay based methods are effective for continual learning without storing history data, but adapting them for this setting is challenging. By analyzing the behaviors of clients during training, we find that the unstable training process caused by distributed training on non-IID data leads to a notable performance degradation. To address this problem, we propose our FedCIL model with two simple but effective solutions: model consolidation and consistency enforcement. Our experimental results on multiple benchmark datasets demonstrate that our method significantly outperforms baselines.
Likelihood Ratio Tests for Seasonal Unit Roots

This paper proposes regression‐based likelihood ratio or F tests for the seasonal unit root hypothesis which fully incorporate the implicit restrictions on the parameters associated with the deterministics. These statistics are similar both exactly and asymptotically with respect to initial values and seasonal drift parameters. The limiting representations of the statistics are presented for a general seasonal aspect. These limiting representations allow those for other scenarios concerning the deterministics to be simply obtained and provide an explanation for the similarity between critical values in apparently quite different cases of interest. We re‐examine the seasonal unit root properties of the logarithm of monthly seasonally unadjusted real industrial production in Canada.
Toward Fairness in Text Generation via Mutual Information Minimization based on Importance Sampling

Pretrained language models (PLMs), such as GPT2, have achieved remarkable empirical performance in text generation tasks. However, pretrained on large-scale natural language corpora, the generated text from PLMs may exhibit social bias against disadvantaged demographic groups. To improve the fairness of PLMs in text generation, we propose to minimize the mutual information between the semantics in the generated text sentences and their demographic polarity, i.e., the demographic group to which the sentence is referring. In this way, the mentioning of a demographic group (e.g., male or female) is encouraged to be independent from how it is described in the generated text, thus effectively alleviating the social bias. Moreover, we propose to efficiently estimate the upper bound of the above mutual information via importance sampling, leveraging a natural language corpus. We also propose a distillation mechanism that preserves the language modeling ability of the PLMs after debiasing. Empirical results on real-world benchmarks demonstrate that the proposed method yields superior performance in term of both fairness and language modeling ability.
Velocity Coupling from Pulse Test Firings

There has been much dispute about triggering of non-linear combustion instability(CI). The norm has been to use ad hoc models to include second order combustion terms most notably velocity coupling. This term arose due to classical linear analyses predicting stable motors necessitating another driving function. However there is no practical universally accepted experimental method that can obtain the velocity coupling response function. The solid rocket motor (SRM) designer is left to his own devices as to what the contribution of velocity coupling may be. This has led to much confusion and rocket motors that have been predicted stable but in practice were not. Flandro and co workers provide an alternative interpretation into the cause of non-linear combustion instability by including rotational terms into the linear analysis. This has been shown to improve the linear stability prediction by predicting unstable systems i.e. CI is always present but at low amplitudes growing too slowly to form a steep fronted wave. A pulse in this context needs to be of high enough amplitude to induce rapid growth. No criterion exists to predict such an amplitude. This study uses a newly developed analysis methodology using tubular grain SRMs to investigate both interpretations i.e. a linearly stable system with velocity coupling or linearly unstable system. This method when combined with response data can be used to obtain a velocity coupling response function from pulse tests. From this analysis it was possible to obtain reasonable velocity coupling response function. When using these constants to predict it was possible to predict a trigger point. Interestingly this point changes significantly with the initial higher order mode amplitudes. It then becomes vital to understand the likely wave structure of a pulse and what influence it has on the triggering amplitude. The system dynamics did not follow that of the experimental data but this may be due to truncation errors and assumptions made in the mathematical model. However the limiting amplitude is predicted correctly. The Flandro interpretation can be evaluated using this new method and applying a systematic study to evaluate criteria for the triggering amplitude. The linearly unstable values seem to capture the system dynamics better. However, in theory any pulse should cause CI. Regardless whether it is true or perceived triggering there seems to be an amplitude value at which non-linear CI is induced. The boundary layer pumping term here seems to be the key to interpreting triggering. The boundary layer acts as a piston that drives the acoustic oscillations when in phase. Thus for non-linear CI to occur the pulse must be of such an amplitude to cause the boundary layer to oscillate and maintain a high enough amplitude until the boundary layer starts to oscillate in phase. Regardless of interpretation this method shows great promise in allowing SRM designers to obtain quantitative response data and investigate triggering quickly and cheaply.
Energy Partition of Thermal and Nonthermal Particles in Magnetic Reconnection

Magnetic reconnection has long been known to be the most important mechanism for quick conversion of magnetic field energy into plasma kinetic energy. In addition, energy dissipation by reconnection has gained attention not only as a plasma heating mechanism, but also as a plasma mechanism for accelerating nonthermal particles. However, the energy partitioning of thermal and nonthermal plasmas during magnetic reconnection is not understood. Here, we studied energy partition as a function of plasma sheet temperature and guide magnetic field. In relativistic reconnection with an antiparallel magnetic field or a weak guide magnetic field, it was found that the nonthermal energy density can occupy more than 90% of the total kinetic plasma energy density, but strengthening the guide magnetic field suppresses the efficiency of the nonthermal particle acceleration. In nonrelativistic reconnection for an antiparallel magnetic field, most dissipated magnetic field energy is converted into thermal plasma heating. For a weak guide magnetic field with a moderate value, however, the nonthermal particle acceleration efficiency was enhanced, but strengthening the guide field beyond the moderate value suppresses the efficiency.
Revisiting Self-Training with Regularized Pseudo-Labeling for Tabular Data

Recent progress in semi- and self-supervised learning has caused a rift in the long-held belief about the need for an enormous amount of labeled data for machine learning and the irrelevancy of unlabeled data. Although it has been successful in various data, there is no dominant semi- and self-supervised learning method that can be generalized for tabular data (i.e. most of the existing methods require appropriate tabular datasets and architectures). In this paper, we revisit self-training which can be applied to any kind of algorithm including the most widely used architecture, gradient boosting decision tree, and introduce curriculum pseudo-labeling (a state-of-the-art pseudo-labeling technique in image) for a tabular domain. Furthermore, existing pseudo-labeling techniques do not assure the cluster assumption when computing confidence scores of pseudo-labels generated from unlabeled data. To overcome this issue, we propose a novel pseudo-labeling approach that regularizes the confidence scores based on the likelihoods of the pseudo-labels so that more reliable pseudo-labels which lie in high density regions can be obtained. We exhaustively validate the superiority of our approaches using various models and tabular datasets.
Navigating the Grey Area: Expressions of Overconfidence and Uncertainty in Language Models

Despite increasingly ﬂuent, relevant, and coherent language generation, major gaps remain between how humans and machines use language. We argue that a key dimension that is missing from our understanding of language models (LMs) is the model’s ability to interpret and generate expressions of uncertainty . Whether it be the weatherperson announcing a chance of rain or a doctor giving a diagnosis, information is often not black-and-white and expressions of uncertainty provide nuance to support human-decision making. The increasing deployment of LMs in the wild motivates us to investigate whether LMs are capable of interpreting expressions of uncertainty and how LMs’ behaviors change when learning to emit their own expressions of uncertainty. When injecting expressions of uncertainty into prompts (e.g., "I think the answer is."), we discover that GPT3’s generations vary upwards of 80% in accuracy based on the expression used. We analyze the linguistic characteristics of these expressions and ﬁnd a drop in accuracy when naturalistic expressions of certainty are present. We ﬁnd similar effects when teaching models to emit their own expressions of uncertainty, where model calibration suffers when teaching models to emit certainty rather than un certainty. Together, these results highlight the challenges of building LMs that interpret and generate trustworthy expressions of uncertainty.
Why Do Facial Deepfake Detectors Fail?

Recent rapid advancements in deepfake technology have allowed the creation of highly realistic fake media, such as video, image, and audio. These materials pose significant challenges to human authentication, such as impersonation, misinformation, or even a threat to national security. To keep pace with these rapid advancements, several deepfake detection algorithms have been proposed, leading to an ongoing arms race between deepfake creators and deepfake detectors. Nevertheless, these detectors are often unreliable and frequently fail to detect deepfakes. This study highlights the challenges they face in detecting deepfakes, including (1) the pre-processing pipeline of artifacts and (2) the fact that generators of new, unseen deepfake samples have not been considered when building the defense models. Our work sheds light on the need for further research and development in this field to create more robust and reliable detectors.
Higher-order correction to weak-field lensing of an Ellis-Bronnikov wormhole

The gravitational lensing effect at higher order under weak-field approximation is believed to be important to distinguish black holes and other compact objects such as wormholes. The deflection angle of a generic wormhole is difficult to solve analytically; thus approximation methods are implemented. In this paper, we investigate the weak-field deflection angle of a specific wormhole, the Ellis-Bronnikov wormhole, up to the 1/b^4 order. We use different approximation formalisms, study their precision at 1/b^4 order by a comparison to a purely numerical result, and finally rank these formalisms by their accuracy. Moreover, we find that certain formalisms are sensitive to the choice of coordinate system; thus it is important to choose the coordinate system appropriately for the evaluating of lensing physics.
Early Star Charts of the Dutch East India Company

As the European maritime powers expanded their reach beyond north Atlantic coastal waters to distant lands as far away as the East Indies, access to a practical means of maritime navigation in the southern hemisphere became imperative. The first few voyages undertaken by the Dutch East India Company and its predecessor explicitly aimed at compiling star charts and constellations that were only visible south of the Equator, as practical navigation aids. The oldest known star atlas of southern constellations was published in 1603 by Frederick de Houtman. Controversies have plagued de Houtman's astronomical credentials from their inception, however, with contemporaries variously attributing the early southern star charts to Pieter Dirkszoon Keyser, de Houtman, or even to their tutor Petrus Plancius. The balance of available evidence suggests that Keyser initially led the astronomical observing campaign, ably assisted by de Houtman. Upon Keyser's untimely death, de Houtman embraced a leading role in compiling astronomical observations for maritime navigation purposes, whereas Plancius most probably led the delineation of the 12 new southern constellations that soon became part and parcel of the nautical consciousness.
Near braces and p$p$‐deformed braided groups

Motivated by recent findings on the derivation of parametric noninvolutive solutions of the Yang–Baxter equation, we reconstruct the underlying algebraic structures, called near braces. Using the notion of the near braces we produce new multi‐parametric, nondegenerate, noninvolutive solutions of the set‐theoretic Yang–Baxter equation. These solutions are generalizations of the known ones coming from braces and skew braces. Bijective maps associated to the inverse solutions are also constructed. Furthermore, we introduce the generalized notion of ‐deformed braided groups and ‐braidings and we show that every ‐braiding is a solution of the braid equation. We also show that certain multi‐parametric maps within the near braces provide special cases of ‐braidings.
A Light-Weight Deep Learning Model for Remote Sensing Image Classification

In this paper, we present a high-performance and light-weight deep learning model for Remote Sensing Image Classification (RSIC), the task of identifying the aerial scene of a remote sensing image. To this end, we first evaluate various benchmark convolutional neural network (CNN) architectures: MobileNet V1/V2, ResNet 50/151V2, InceptionV3/InceptionResNetV2, EfficientNet B0/B7, DenseNet 121/201, ConNeXt Tiny/Large. Then, the best performing models are selected to train a compact model in a teacher-student arrangement. The knowledge distillation from the teacher aims to achieve high performance with significantly reduced complexity. By conducting extensive experiments on the NWPU-RESISC45 benchmark, our proposed teacher and student models outper-forms the state-of-the-art systems, and has potential to be applied on a wide range of edge devices.
Suppression of one-dimensional weak localization by band asymmetry

We investigate disorder-induced localization in metals that break time-reversal and inversion symmetries through their energy dispersion, $\epsilon_{k}\neq\epsilon_{-k}$, but lack Berry phases. In the perturbative regime of disorder, we show that weak localization is suppressed due to a mismatch of the Fermi velocities of left and right movers. To substantiate this analytical result, we perform quench numerics on chains shorter than the Anderson localization length -- the latter computed and verified to be finite using the recursive Green's function method -- and find a sharp rise in the saturation value of the participation ratio due to band asymmetry, indicating a tendency to delocalize. Interestingly, for weak disorder strength $\eta$, we see a better fit to the scaling behavior $\xi\propto1/\eta^{2}$ for asymmetric bands than conventional symmetric ones.
NbN films with high kinetic inductance for high-quality compact superconducting resonators

Niobium nitride (NbN) is a particularly promising material for quantum technology applications, as entails the degree of reproducibility necessary for large-scale of superconducting circuits. We demonstrate that resonators based on NbN thin films present a one-photon internal quality factor above 10$^5$ maintaining a high impedance (larger than 2k$\Omega$), with a footprint of approximately 50x100 $\mu$m$^2$ and a self-Kerr nonlinearity of few tenths of Hz. These quality factors, mostly limited by losses induced by the coupling to two-level systems, have been maintained for kinetic inductances ranging from tenths to hundreds of pH/square. We also demonstrate minimal variations in the performance of the resonators during multiple cooldowns over more than nine months. Our work proves the versatility of niobium nitride high-kinetic inductance resonators, opening perspectives towards the fabrication of compact, high-impedance and high-quality multimode circuits, with sizable interactions.
Exact computation of tail probability of hypergeometric distribution and its application to Quantum Key Distribution

: Tomamichel et al. improved a key rate of a Quantum Key Distribution (QKD) protocol with ﬁnite number of qubits. To compute the key rate, we need to estimate a phase error rate, which obey a hypergeometric distribution. They did not compute directly the tail probability of the hypergeometric distribution to obtain the estimate of the error rate, and they used its upper bound instead. We improve the key rate of the QKD by exact numerical computation of the tail probability. We also propose a method to decrease the computation time for exact numerical computation.
Using Artificial Neural Network to Predict Body Weights of Rabbits

In this (modest) study, we developed artificial neural network (ANN) models for predicting body weight using various independent (input) variables in eight-week old New Zealand white purebred and crossbred rabbits. From the whole data sets of similar age groups, 75 percent were used to train the neural network model and 25 percent were used to test the effectiveness of the model. Five predictor variables were used viz, breed, sex, heart girth, body length and height at wither as input variables and body weight was considered as dependent variable from the model. The ANN used was multilayer feed forward network with back propagation of error for efficient learning. Our ANN models (with R2 = 0.68 at ten thousand iterations, and R2 = 0.71 one million iterations) performed better than traditional multivariate linear regression (MLR) models (R2 = 0.66) indicating that the ANN models were able to more accurately capture how the variations in input variables explained the variations in body weight. It is concluded that ANN models are more powerful than MLR models in predicting animals’ body weight. Nonetheless, we recognize that fitting an ANN model requires more computation resources than fitting a tradition MLR model but the benefits of its accuracy outweigh any demerit from the associated computation overhead.
Conserving the wrong variables in gas dynamics: A Riemann solution with singular shocks

We consider a system of two equations derived from isentropic gas dynamics with no classical Riemann solutions. We show existence of unbounded self-similar solutions (singular shocks) of the Dafermos regularization of the system. Our approach is based on the blowing-up approach of geometric singular perturbation theory.
Estimation of differential Doppler shifts

If the acoustic signal radiated by a moving source is observed at two or more locations, the received signals exhibit differential Doppler shifts which provide information about source motion. This paper calculates minimum mean‐square errors for the estimates of differential Doppler shifts, which can be obtained in a given observation interval. Both Gaussian and sinusoidal signals are considered. The noise is assumed to be Gaussian and independent from sensor to sensor. Dependence of the estimation errors on observation time, signal‐to‐noise ratio, and size of the receiving array are studied. The estimation of differential Dopplers is found to be uncoupled from the estimation of differential delays and from the estimation of signal parameters, such as center frequency and bandwidth. A comparison is made between two possible procedures of differential Doppler estimation: coherent processing of the signals received at two sensors and subtraction of separate frequency estimates obtained at each sensor. The t.