Miracle-dz/UniFAR-FTUs huggingface dataset:
Self-Explaining Structures Improve NLP Models

Existing approaches to explaining deep learning models in NLP usually suffer from two major drawbacks: (1) the main model and the explaining model are decoupled: an additional probing or surrogate model is used to interpret an existing model, and thus existing explaining tools are not self-explainable; (2) the probing model is only able to explain a model's predictions by operating on low-level features by computing saliency scores for individual words but are clumsy at high-level text units such as phrases, sentences, or paragraphs. To deal with these two issues, in this paper, we propose a simple yet general and effective self-explaining framework for deep learning models in NLP. The key point of the proposed framework is to put an additional layer, as is called by the interpretation layer, on top of any existing NLP model. This layer aggregates the information for each text span, which is then associated with a specific weight, and their weighted combination is fed to the softmax function for the final prediction. The proposed model comes with the following merits: (1) span weights make the model self-explainable and do not require an additional probing model for interpretation; (2) the proposed model is general and can be adapted to any existing deep learning structures in NLP; (3) the weight associated with each text span provides direct importance scores for higher-level text units such as phrases and sentences. We for the first time show that interpretability does not come at the cost of performance: a neural model of self-explaining features obtains better performances than its counterpart without the self-explaining nature, achieving a new SOTA performance of 59.1 on SST-5 and a new SOTA performance of 92.3 on SNLI.
Temperature Dependence of the Resistivity in Semimetals of the Bismuth Type

The resistivity of semi-metallic bismuth-antimony alloys and tellurium- or tin-doped bismuth specimens is studied in a temperature range from 1.8 to 77 K. Typical dependences of the specific resistivity ϱ(T) as a function of Fermi energy are conceivable on the basis of an electron-phonon scattering theory. In einem Temperaturbereich von 1,8 bis 77 K wird der Wird der Widerstand halbmetallischer Wismut-Antimonlegierungen und tellur-bzw. zinndotierter Wismutproben untersucht. Typische Abhangigkeiten des spezifischen Widerstandes ϱ(T) als Funktion der Fermienergie konnen im Rahmen einer Elektron-Phonon-Streutheorie verstanden werden.
Phenethyl Isothiocyanate Protects against High Fat/Cholesterol Diet-Induced Obesity and Atherosclerosis in C57BL/6 Mice

This study concerns obesity-related atherosclerosis, hyperlipidemia, and chronic inflammation. We studied the anti-obesity and anti-atherosclerosis effects of phenethyl isothiocyanate (PEITC) and explored their underlying mechanisms. We established an animal model of high fat/cholesterol-induced obesity in C57BL/6 mice fed for 13 weeks. We divided the mice into five groups: control (CON), high fat/cholesterol (HFCD), HFCD with 3 mg/kg/day gallic acid (HFCD + G), and HFCD with PEITC (30 and 75 mg/kg/day; HFCD + P30 and P75). The body weight, total cholesterol, and triglyceride were significantly lower in the HFCD + P75 group than in the HFCD group. Hepatic lipid accumulation and atherosclerotic plaque formation in the aorta were significantly lower in both HFCD + PEITC groups than in the HFCD group, as revealed by hematoxylin and eosin (H&E) staining. To elucidate the mechanism, we identified the expression of genes related to inflammation, reverse cholesterol transport, and lipid accumulation pathway in the liver. The expression levels of peroxisome proliferator activated receptor gamma (PPARγ), liver-X-receptor α (LXR-α), and ATP binding cassette subfamily A member 1 (ABCA1) were increased, while those of scavenger receptor A (SR-A1), cluster of differentiation 36 (CD36), and nuclear factor-kappa B (NF-κB) were decreased in the HFCD + P75 group compared with those in the HFCD group. Moreover, PEITC modulated H3K9 and H3K27 acetylation, H3K4 dimethylation, and H3K27 di-/trimethylation in the HFCD + P75 group. We, therefore, suggest that supplementation with PEITC may be a potential candidate for the treatment and prevention of atherosclerosis and obesity.
Model-Free Neural Counterfactual Regret Minimization With Bootstrap Learning

Counterfactual regret minimization (CFR) has achieved many fascinating results in solving large-scale imperfect information games (IIGs). Neural network approximation CFR (neural CFR) is one of the promising techniques that can reduce computation and memory consumption by generalizing decision information between similar states. Current neural CFR algorithms have to approximate cumulative regrets. However, efficient and accurate approximation in a large-scale IIG is still a tough challenge. In this article, a new CFR variant, recursive CFR (ReCFR), is proposed. In ReCFR, recursive substitute values (RSVs) are learned and used to replace cumulative regrets. It is proven that ReCFR can converge to a Nash equilibrium at a rate of $O({1}/{\sqrt{T}})$. Based on ReCFR, a new model-free neural CFR with bootstrap learning, neural ReCFR-B, is proposed. Due to the recursive and noncumulative nature of RSVs, neural ReCFR-B has lower variance training targets than other neural CFRs. Experimental results show that neural ReCFR-B is competitive with the state-of-the-art neural CFR algorithms at a much lower training cost.
Phonetic Posteriorgrams based Many-to-Many Singing Voice Conversion via Adversarial Training

This paper describes an end-to-end adversarial singing voice conversion (EA-SVC) approach. It can directly generate arbitrary singing waveform by given phonetic posteriorgram (PPG) representing content, F0 representing pitch, and speaker embedding representing timbre, respectively. Proposed system is composed of three modules: generator $G$, the audio generation discriminator $D_{A}$, and the feature disentanglement discriminator $D_F$. The generator $G$ encodes the features in parallel and inversely transforms them into the target waveform. In order to make timbre conversion more stable and controllable, speaker embedding is further decomposed to the weighted sum of a group of trainable vectors representing different timbre clusters. Further, to realize more robust and accurate singing conversion, disentanglement discriminator $D_F$ is proposed to remove pitch and timbre related information that remains in the encoded PPG. Finally, a two-stage training is conducted to keep a stable and effective adversarial training process. Subjective evaluation results demonstrate the effectiveness of our proposed methods. Proposed system outperforms conventional cascade approach and the WaveNet based end-to-end approach in terms of both singing quality and singer similarity. Further objective analysis reveals that the model trained with the proposed two-stage training strategy can produce a smoother and sharper formant which leads to higher audio quality.
3D optical/CT as a preclinical companion imaging platform for glioblastoma drug development

Abstract Multimodality 3D Optical Imaging (OI)/CT has the potential to play a major role in drug development for glioblastomas (GBM), as it is an accessible preclinical method. To demonstrate the potential of 3D OI/CT to visualize orthotopic GBM implantation, we labeled GBM cells with Cy7 and imaged their location using 3D OI/CT. To confirm the accuracy of the spatial localization and demonstrate the ability to image locoregionally delivered therapies, we labeled mouse albumin with Cy7 (Cy7ALB) and delivered it via locoregional infusion 1 mm or 3 mm into the brain and demonstrated correlation of signal between the 3D OI/CT and post necropsy brain slices. In addition, we demonstrated the potential of systemically delivered Cy7ALB contrast to detect blood–brain barrier (BBB) permeability caused by orthotopic GBMs using 3D OI/CT. We also tested the potential of 3D OI/CT to assess focal BBB permeability induced by high intensity focused ultrasound (HIFU), a methodology being used in clinical trials to noninvasively permeabilize the BBB for systemic therapeutic delivery to GBM. We demonstrated the ability of systemic Cy7ALB contrast together with 3D OI/CT to accurately assess real-time HIFU-induced BBB permeability, which correlated to post necropsy imaging of brains. Furthermore, we demonstrated that 3D OI/CT can also image the therapeutic distribution of a Cy7-labeled anti-PD-1 antibody, a prototype translational antibody therapy. We successfully imaged real-time antibody distribution after HIFU-induced BBB permeability, which correlated with post necropsy Cy7 signal and translational PET imaging after injection of [89Zr] anti-PD-1 antibody. Thus, we demonstrated the broad potential of using 3D OI/CT as an accessible preclinical tool to develop anti-GBM therapies.
Associations of body mass index and waist circumference with risk of Guillain-Barré syndrome in women and men: A prospective analysis of three cohort studies

Background The association of body mass index (BMI) and waist circumference (WC) with risk of Guillain-Barré syndrome (GBS) has been inconsistent in previous studies. Methods We examined the associations of BMI and WC in relation to risk of GBS among 252,980 participants from the Nurses’ Health Study (NHS), NHS-II, and the Health Professional Follow-up Study (HPFS). BMI and WC were assessed by self-reported questionnaire, and GBS cases were self-reported. Results We documented 328 incident GBS cases during a total of 5,422,788 person years of follow-up. Compared to participants with BMI<25kg/m2, the multivariate pooled hazard ratio (HR) of GBS was 1.34 (95% CI: 1.04, 1.73) for overweight participants (25kg/m2≤BMI<30 kg/m2), and 1.68 (95% CI: 1.21, 2.35) for obese participants (BMI≥30 kg/m2) (P for trend = 0.001). Compared to participants with normal WC (<35 inches for women and <40 inches for men), the HR of GBS was 1.55 (95% CI: 1.10, 2.18) for participants with high WC (≥35 inches for women, and ≥40 inches for men). The positive associations of BMI and WC with risk of GBS were attenuated to null after mutually adjusting for BMI and WC. Joint analysis showed that the HR was 1.84 (95% CI: 1.27, 2.66) for participants with high WC and BMI≥25 kg/m2 in comparison to participants with normal WC and BMI<25kg/m2. Conclusion These data from large cohorts showed that higher BMI and WC jointly were associated with higher risk of GBS. Our study highlighted the importance of maintaining a normal body weight and waist circumference in prevention of GBS.
Second‐order semi‐parametric inference for multivariate log Gaussian Cox processes

This paper introduces a new approach to inferring the second‐order properties of a multivariate log Gaussian Cox process (LGCP) with a complex intensity function. We assume a semi‐parametric model for the multivariate intensity function containing an unspecified complex factor common to all types of points. Given this model, we construct a second‐order conditional composite likelihood to infer the pair correlation and cross pair correlation functions of the LGCP. Crucially this likelihood does not depend on the unspecified part of the intensity function. We also introduce a cross‐validation method for model selection and an algorithm for regularized inference that can be used to obtain sparse models for cross pair correlation functions. The methodology is applied to simulated data as well as data examples from microscopy and criminology. This shows how the new approach outperforms existing alternatives where the intensity functions are estimated non‐parametrically.
Reconstructing cellular automata rules from observations at nonconsecutive times

Recent experiments have shown that a deep neural network can be trained to predict the action of t steps of Conway's Game of Life automaton given millions of examples of this action on random initial states. However, training was never completely successful for t>1, and even when successful, a reconstruction of the elementary rule (t=1) from t>1 data is not within the scope of what the neural network can deliver. We describe an alternative network-like method, based on constraint projections, where this is possible. From a single data item this method perfectly reconstructs not just the automaton rule but also the states in the time steps it did not see. For a unique reconstruction, the size of the initial state need only be large enough that it and the t-1 states it evolves into contain all possible automaton input patterns. We demonstrate the method on 1D binary cellular automata that take inputs from n adjacent cells. The unknown rules in our experiments are not restricted to simple rules derived from a few linear functions on the inputs (as in Game of Life), but include all 2^{2^{n}} possible rules on n inputs. Our results extend to n=6, for which exhaustive rule-search is not feasible. By relaxing translational symmetry in space and also time, our method is attractive as a platform for the learning of binary data, since the discreteness of the variables does not pose the same challenge it does for gradient-based methods.
A Concern Analysis of FOMC Statements Comparing The Great Recession and The COVID-19 Pandemic

It is important and informative to compare and contrast major economic crises in order to confront novel and unknown cases such as the COVID-19 pandemic. The 2006 Great Recession and then the 2019 pandemic have a lot to share in terms of unemployment rate, consumption expenditures, and interest rates set by Federal Reserve. In addition to quantitative historical data, it is also interesting to compare the contents of Federal Reserve statements for the period of these two crises and find out whether Federal Reserve cares about similar concerns or there are some other issues that demand separate and unique monetary policies. This paper conducts an analysis to explore the Federal Reserve concerns as expressed in their statements for the period of 2005 to 2020. The concern analysis is performed using natural language processing (NLP) algorithms and a trend analysis of concern is also presented. We observe that there are some similarities between the Federal Reserve statements issued during the Great Recession with those issued for the 2019 COVID-19 pandemic.
Waveguide-Based Fluorescent Immunosensor for the Simultaneous Detection of Carbofuran and 3-Hydroxy-Carbofuran

Carbofuran (CBF) is an efficient and broad-spectrum insecticide. As testing indicators for water quality and agricultural products, CBF and its metabolite 3-hydroxy-carbofuran (3-OH-CBF) are regulated by many countries. The detection of CBF and 3-OH-CBF is of great importance for the environment and human health. However, an immunosensor detection method for the simultaneous analysis of CBF and 3-OH-CBF remains unavailable. Herein, we report a waveguide-based fluorescent immunosensor for detecting CBF and 3-OH-CBF, synchronously. The immunosensor is based on a broad-spectrum monoclonal antibody with high binding affinity against CBF and 3-OH-CBF. The linear detection ranges for CBF and 3-OH-CBF are 0.29–2.69 and 0.12–4.59 μg/L, with limits of detection of 0.13 μg/L for CBF and 0.06 μg/L for 3-OH-CBF, respectively. The whole detection process for each cycle is approximately 30 min. The results show a good application prospect for the rapid detection of CBF and 3-OH-CBF in water or agricultural products.
Source location on multilayer networks

Nowadays it is not uncommon to have to deal with dissemination on multi-layered networks and often finding the source of said propagation can be a crucial task. In this paper we tackle this exact problem with a maximum likelihood approach that we extend to be operational on multi-layered graphs. We test our method for source location estimation on synthetic networks and outline its potential strengths and limitations. We also observe some non-trivial and perhaps surprising phenomena where the more of the system one observes the worse the results become whereas increased problem complexity in the form of more layers can actually improve our performance.
Cross-correlation based discriminant criterion for channel selection in motor imagery BCI systems

Objective. Many electroencephalogram (EEG)-based brain-computer interface (BCI) systems use a large amount of channels for higher performance, which is time-consuming to set up and inconvenient for practical applications. Finding an optimal subset of channels without compromising the performance is a necessary and challenging task. Approach. In this article, we proposed a cross-correlation based discriminant criterion (XCDC) which assesses the importance of a channel for discriminating the mental states of different motor imagery (MI) tasks. Channels are ranked and selected according to the proposed criterion. The efficacy of XCDC is evaluated on two MI EEG datasets. Main results. On the two datasets, the proposed method reduces the channel number from 71 and 15 to under 18 and 11 respectively without compromising the classification accuracy on unseen data. Under the same constraint of accuracy, the proposed method requires fewer channels than existing channel selection methods based on Pearson’s correlation coefficient and common spatial pattern. Visualization of XCDC shows consistent results with neurophysiological principles. Significance. This work proposes a quantitative criterion for assessing and ranking the importance of EEG channels in MI tasks and provides a practical method for selecting the ranked channels in the calibration phase of MI BCI systems, which alleviates the computational complexity and configuration difficulty in the subsequent steps, leading to real-time and more convenient BCI systems.
SuperOCR: A Conversion from Optical Character Recognition to Image Captioning

Optical Character Recognition (OCR) has many real world applications. The existing methods normally detect where the characters are, and then recognize the character for each detected location. Thus the accuracy of characters recognition is impacted by the performance of characters detection. In this paper, we propose a method for recognizing characters without detecting the location of each character. This is done by converting the OCR task into an image captioning task. One advantage of the proposed method is that the labeled bounding boxes for the characters are not needed during training. The experimental results show the proposed method outperforms the existing methods on both the license plate recognition and the watermeter character recognition tasks. The proposed method is also deployed into a low-power (300mW) CNN accelerator chip connected to a Raspberry Pi 3 for on-device applications.
Integrable Nonparametric Flows

We introduce a method for reconstructing an infinitesimal normalizing flow given only an infinitesimal change to a (possibly unnormalized) probability distribution. This reverses the conventional task of normalizing flows -- rather than being given samples from a unknown target distribution and learning a flow that approximates the distribution, we are given a perturbation to an initial distribution and aim to reconstruct a flow that would generate samples from the known perturbed distribution. While this is an underdetermined problem, we find that choosing the flow to be an integrable vector field yields a solution closely related to electrostatics, and a solution can be computed by the method of Green's functions. Unlike conventional normalizing flows, this flow can be represented in an entirely nonparametric manner. We validate this derivation on low-dimensional problems, and discuss potential applications to problems in quantum Monte Carlo and machine learning.
Sparse Semi-Supervised Action Recognition with Active Learning

Current state-of-the-art methods for skeleton-based action recognition are supervised and rely on labels. The reliance is limiting the performance due to the challenges involved in annotation and mislabeled data. Unsupervised methods have been introduced, however, they organize sequences into clusters and still require labels to associate clusters with actions. In this paper, we propose a novel approach for skeleton-based action recognition, called SESAR, that connects these approaches. SESAR leverages the information from both unlabeled data and a handful of sequences actively selected for labeling, combining unsupervised training with sparsely supervised guidance. SESAR is composed of two main components, where the first component learns a latent representation for unlabeled action sequences through an Encoder-Decoder RNN which reconstructs the sequences, and the second component performs active learning to select sequences to be labeled based on cluster and classification uncertainty. When the two components are simultaneously trained on skeleton-based action sequences, they correspond to a robust system for action recognition with only a handful of labeled samples. We evaluate our system on common datasets with multiple sequences and actions, such as NW UCLA, NTU RGB+D 60, and UWA3D. Our results outperform standalone skeleton-based supervised, unsupervised with cluster identification, and active-learning methods for action recognition when applied to sparse labeled samples, as low as 1% of the data.
SSGD: A safe and efficient method of gradient descent

With the vigorous development of artificial intelligence technology, various engineering technology applications have been implemented one after another. The gradient descent method plays an important role in solving various optimization problems, due to its simple structure, good stability, and easy implementation. However, in multinode machine learning system, the gradients usually need to be shared, which will cause privacy leakage, because attackers can infer training data with the gradient information. In this paper, to prevent gradient leakage while keeping the accuracy of the model, we propose the super stochastic gradient descent approach to update parameters by concealing the modulus length of gradient vectors and converting it or them into a unit vector. Furthermore, we analyze the security of super stochastic gradient descent approach and demonstrate that our algorithm can defend against the attacks on the gradient. Experiment results show that our approach is obviously superior to prevalent gradient descent approaches in terms of accuracy, robustness, and adaptability to large-scale batches. Interestingly, our algorithm can also resist model poisoning attacks to a certain extent.
Perfectly Matched Layers for nonlocal Helmholtz equations Part II: higher dimensions

Perfectly matched layers (PMLs) are formulated and numerically applied to nonlocal Helmholtz equations in one and two dimensions. In one dimension, we give the PML modifications for the nonlocal Helmholtz equation with general kernels and show its effectiveness theoretically in some sense. In two dimensions, we give the PML modifications in both Cartesian coordinates and polar coordinates. Based on the PML modifications, nonlocal Helmholtz equations are truncated in one and two dimensional spaces, and asymptotic compatibility schemes are introduced to solve the resulting truncated problems. Finally, numerical examples are provided to study the "numerical reflections" by PMLs and verify the effectiveness and validation of our nonlocal PML strategy.