Miracle-dz/UniFAR-FTUs huggingface dataset:
Feasible Actor-Critic: Constrained Reinforcement Learning for Ensuring Statewise Safety

The safety constraints commonly used by existing safe reinforcement learning (RL) methods are defined only on expectation of initial states, but allow each certain state to be unsafe, which is unsatisfying for real-world safety-critical tasks. In this paper, we introduce the feasible actor-critic (FAC) algorithm, which is the first model-free constrained RL method that considers statewise safety, e.g, safety for each initial state. We claim that some states are inherently unsafe no matter what policy we choose, while for other states there exist policies ensuring safety, where we say such states and policies are feasible. By constructing a statewise Lagrange function available on RL sampling and adopting an additional neural network to approximate the statewise Lagrange multiplier, we manage to obtain the optimal feasible policy which ensures safety for each feasible state and the safest possible policy for infeasible states. Furthermore, the trained multiplier net can indicate whether a given state is feasible or not through the statewise complementary slackness condition. We provide theoretical guarantees that FAC outperforms previous expectation-based constrained RL methods in terms of both constraint satisfaction and reward optimization. Experimental results on both robot locomotive tasks and safe exploration tasks verify the safety enhancement and feasibility interpretation of the proposed method.
Model-Assisted Inference for Covariate-Specific Treatment Effects with High-dimensional Data

Covariate-specific treatment effects (CSTEs) represent heterogeneous treatment effects across subpopulations defined by certain selected covariates. In this article, we consider marginal structural models where CSTEs are linearly represented using a set of basis functions of the selected covariates. We develop a new approach in high-dimensional settings to obtain not only doubly robust point estimators of CSTEs, but also model-assisted confidence intervals, which are valid when a propensity score model is correctly specified but an outcome regression model may be misspecified. With a linear outcome model and subpopulations defined by discrete covariates, both point estimators and confidence intervals are doubly robust for CSTEs. In contrast, confidence intervals from existing high-dimensional methods are valid only when both the propensity score and outcome models are correctly specified. We establish asymptotic properties of the proposed point estimators and the associated confidence intervals. We present simulation studies and empirical applications which demonstrate the advantages of the proposed method compared with competing ones.
Renormalization-group study of Bose polarons

We study the properties of a single impurity in a dilute Bose gas, a Bose polaron, using the functional renormalization group. We use an ansatz for the effective action motivated by a derivative expansion, and we compute the energies of the attractive and repulsive branches of excitations in both two and three spatial dimensions. Three-body correlations play an important role in the attractive branch, and we account for those by including three-body couplings between two bath bosons and the impurity. Our calculations compare very favorably with state-of-the-art experimental measurements and numerical simulations.
Electronic structure of 
SrTi1−xVxO3
 films studied by 
in situ
 photoemission spectroscopy: Screening for a transparent electrode material

This study investigated the electronic structure of SrTi1-xVxO3 (STVO) thin films, which are solid solutions of strongly correlated transparent conductive oxide (TCO) SrVO3 and oxide semiconductor SrTiO3, using in situ photoemission spectroscopy. STVO is one of the most promising candidates for correlated-metal TCO because it has the capability of optimizing the performance of transparent electrodes by varying x. Systematic and significant spectral changes were found near the Fermi level (EF) as a function of x, while the overall electronic structure of STVO is in good agreement with the prediction of band structure calculations. As x decreases from 1.0, spectral weight transfer occurs from the coherent band near EF to the incoherent states (lower Hubbard band) around 1.0–1.5 eV. Simultaneously, a pseudogap is formed at EF, indicating a significant reduction in quasiparticle spectral weight within close vicinity of EF. This pseudogap seems to evolve into an energy gap at x = 0.4, suggesting the occurrence of a composition-driven metal–insulator transition. From angleresolved photoemission spectroscopic results, the carrier concentration n changes proportionally as a function of x in the metallic range of x = 0.6–1.0. In contrast, the mass enhancement factor, which is proportional to the effective mass (m*), does not change significantly with varying x. These results suggest that the key factor of n/m* in optimizing the performance of correlated-metal TCO is tuned 2 by x, highlighting the potential of STVO to achieve the desired TCO performance in the metallic region. † Present Address: Department of Applied Physics, Osaka University, Suita, Osaka 565-0871, Japan *Author to whom correspondence should be addressed: kumigashira@tohoku.ac. jp
Locally private online change point detection

We study online change point detection problems under the constraint of local differential privacy (LDP) where, in particular, the statistician does not have access to the raw data. As a concrete problem, we study a multivariate nonparametric regression problem. At each time point $t$, the raw data are assumed to be of the form $(X_t, Y_t)$, where $X_t$ is a $d$-dimensional feature vector and $Y_t$ is a response variable. Our primary aim is to detect changes in the regression function $m_t(x)=\mathbb{E}(Y_t |X_t=x)$ as soon as the change occurs. We provide algorithms which respect the LDP constraint, which control the false alarm probability, and which detect changes with a minimal (minimax rate-optimal) delay. To quantify the cost of privacy, we also present the optimal rate in the benchmark, non-private setting. These non-private results are also new to the literature and thus are interesting \emph{per se}. In addition, we study the univariate mean online change point detection problem, under privacy constraints. This serves as the blueprint of studying more complicated private change point detection problems.
Prevent the Language Model from being Overconfident in Neural Machine Translation

The Neural Machine Translation (NMT) model is essentially a joint language model conditioned on both the source sentence and partial translation. Therefore, the NMT model naturally involves the mechanism of the Language Model (LM) that predicts the next token only based on partial translation. Despite its success, NMT still suffers from the hallucination problem, generating fluent but inadequate translations. The main reason is that NMT pays excessive attention to the partial translation while neglecting the source sentence to some extent, namely overconfidence of the LM. Accordingly, we define the Margin between the NMT and the LM, calculated by subtracting the predicted probability of the LM from that of the NMT model for each token. The Margin is negatively correlated to the overconfidence degree of the LM. Based on the property, we propose a Margin-based Token-level Objective (MTO) and a Margin-based Sentence-level Objective (MSO) to maximize the Margin for preventing the LM from being overconfident. Experiments on WMT14 English-to-German, WMT19 Chinese-to-English, and WMT14 English-to-French translation tasks demonstrate the effectiveness of our approach, with 1.36, 1.50, and 0.63 BLEU improvements, respectively, compared to the Transformer baseline. The human evaluation further verifies that our approaches improve translation adequacy as well as fluency.
Modeling Collective Anticipation and Response on Wikipedia

The dynamics of popularity in online media are driven by a combination of endogenous spreading mechanisms and response to exogenous shocks including news and events. However, little is known about the dependence of temporal patterns of popularity on event-related information, e.g. which types of events trigger long-lasting activity. Here we propose a simple model that describes the dynamics around peaks of popularity by incorporating key features, i.e., the anticipatory growth and the decay of collective attention together with circadian rhythms. The proposed model allows us to develop a new method for predicting the future page view activity and for clustering time series. To validate our methodology, we collect a corpus of page view data from Wikipedia associated to a range of planned events, that are events which we know in advance will have a fixed date in the future, such as elections and sport events. Our methodology is superior to existing models in both prediction and clustering tasks. Furthermore, restricting to Wikipedia pages associated to association football, we observe that the specific realization of the event, in our case which team wins a match or the type of the match, has a significant effect on the response dynamics after the event. Our work demonstrates the importance of appropriately modeling all phases of collective attention, as well as the connection between temporal patterns of attention and characteristic underlying information of the events they represent.
Structural Pre-training for Dialogue Comprehension

Pre-trained language models (PrLMs) have demonstrated superior performance due to their strong ability to learn universal language representations from self-supervised pre-training. However, even with the help of the powerful PrLMs, it is still challenging to effectively capture task-related knowledge from dialogue texts which are enriched by correlations among speaker-aware utterances. In this work, we present SPIDER, Structural Pre-traIned DialoguE Reader, to capture dialogue exclusive features. To simulate the dialogue-like features, we propose two training objectives in addition to the original LM objectives: 1) utterance order restoration, which predicts the order of the permuted utterances in dialogue context; 2) sentence backbone regularization, which regularizes the model to improve the factual correctness of summarized subject-verb-object triplets. Experimental results on widely used dialogue benchmarks verify the effectiveness of the newly introduced self-supervised tasks.
OntoED: Low-resource Event Detection with Ontology Embedding

Event Detection (ED) aims to identify event trigger words from a given text and classify it into an event type. Most current methods to ED rely heavily on training instances, and almost ignore the correlation of event types. Hence, they tend to suffer from data scarcity and fail to handle new unseen event types. To address these problems, we formulate ED as a process of event ontology population: linking event instances to pre-defined event types in event ontology, and propose a novel ED framework entitled OntoED with ontology embedding. We enrich event ontology with linkages among event types, and further induce more event-event correlations. Based on the event ontology, OntoED can leverage and propagate correlation knowledge, particularly from data-rich to data-poor event types. Furthermore, OntoED can be applied to new unseen event types, by establishing linkages to existing ones. Experiments indicate that OntoED is more predominant and robust than previous approaches to ED, especially in data-scarce scenarios.
Coarse to Fine Multi-Resolution Temporal Convolutional Network

Temporal convolutional networks (TCNs) are a commonly used architecture for temporal video segmentation. TCNs however, tend to suffer from over-segmentation errors and require additional refinement modules to ensure smoothness and temporal coherency. In this work, we propose a novel temporal encoder-decoder to tackle the problem of sequence fragmentation. In particular, the decoder follows a coarse-to-fine structure with an implicit ensemble of multiple temporal resolutions. The ensembling produces smoother segmentations that are more accurate and better-calibrated, bypassing the need for additional refinement modules. In addition, we enhance our training with a multi-resolution feature-augmentation strategy to promote robustness to varying temporal resolutions. Finally, to support our architecture and encourage further sequence coherency, we propose an action loss that penalizes misclassifications at the video level. Experiments show that our stand-alone architecture, together with our novel feature-augmentation strategy and new loss, outperforms the state-of-the-art on three temporal video segmentation benchmarks.
Killing Two Birds with One Stone: Stealing Model and Inferring Attribute from BERT-based APIs

The advances in pre-trained models (e.g., BERT, XLNET and etc) have largely revolutionized the predictive performance of various modern natural language processing tasks. This allows corporations to provide machine learning as a service (MLaaS) by encapsulating ﬁne-tuned BERT-based models as commercial APIs. However, previous works have discovered a series of vulnerabilities in BERT-based APIs. For example, BERT-based APIs are vulnerable to both model extraction attack and adversarial example transferrability attack. However, due to the high capacity of BERT-based APIs, the ﬁne-tuned model is easy to be overlearned, what kind of information can be leaked from the extracted model remains unknown and is lacking. To bridge this gap, in this work, we ﬁrst present an effective model extraction attack, where the adversary can practically steal a BERT-based API (the target/victim model) by only querying a limited number of queries. We further develop an effective attribute inference attack to expose the sensitive attribute of the training data used by the BERT-based APIs. Our extensive experiments on benchmark datasets under various realistic settings demonstrate the potential vulnerabilities of BERT-based APIs.
Multi-Feature Fusion-based Scene Classification Framework for HSR Images

To realize high-accuracy classification of high spatial resolution (HSR) images, this letter proposes a new multi-feature fusion-based scene classification framework (MF2SCF) by fusing local, global, and color features of HSR images. Specifically, we first extract the local features with the help of image slicing and densely connected convolutional networks (DenseNet), where the outputs of dense blocks in the fine-tuned DenseNet-121 model are jointly averaged and concatenated to describe local features. Second, from the perspective of complex networks (CN), we model a HSR image as an undirected graph based on pixel distance, intensity, and gradient, and obtain a gray-scale image (GSI), a gradient of image (GoI), and three CN-based feature images to delineate global features. To make the global feature descriptor resist to the impact of rotation and illumination, we apply uniform local binary patterns (LBP) on GSI, GoI, and feature images, respectively, and generate the final global feature representation by concatenating spatial histograms. Third, the color features are determined based on the normalized HSV histogram, where HSV stands for hue, saturation, and value, respectively. Finally, three feature vectors are jointly concatenated for scene classification. Experiment results show that MF2SCF significantly improves the classification accuracy compared with state-of-the-art LBP-based methods and deep learning-based methods.
A Random Attention and Utility Model

We generalize the stochastic revealed preference methodology of McFadden and Richter (1990) for finite choice sets to settings with limited consideration. Our approach is nonparametric and requires partial choice set variation. We only impose a monotonicity condition on attention first proposed by Cattaneo et al. (2020) and a stability condition on the marginal distribution of preferences. Our framework is amenable to statistical testing. These new restrictions extend widely known parametric models of consideration.
Centralized Learning of the Distributed Downlink Channel Estimators in FDD Systems using Uplink Data

In this work, we propose a convolutional neural network (CNN) based low-complexity approach for downlink (DL) channel estimation (CE) in frequency division duplex (FDD) systems. In contrast to existing work, we use training data which solely stems from the uplink (UL) domain. This allows to learn the CNN centralized at the base station (BS). After training, the network parameters are offloaded to mobile terminals (MTs) within the coverage area of the BS. The MTs can then obtain channel state information (CSI) of the MIMO channels with the low-complexity CNN estimator. This circumvents the necessity of an infeasible amount of feedback, i.e., acquisition of training data at the user, and the offline training phase at each MT. Numerical results show that the CNN which is trained solely based on UL data performs equally well as the network trained based on DL data. Furthermore, the approach is able to outperform state-ofthe-art CE algorithms.
Properties of the After Kernel

The Neural Tangent Kernel (NTK) is the wide-network limit of a kernel defined using neural networks at initialization, whose embedding is the gradient of the output of the network with respect to its parameters. We study the"after kernel", which is defined using the same embedding, except after training, for neural networks with standard architectures, on binary classification problems extracted from MNIST and CIFAR-10, trained using SGD in a standard way. For some dataset-architecture pairs, after a few epochs of neural network training, a hard-margin SVM using the network's after kernel is much more accurate than when the network's initial kernel is used. For networks with an architecture similar to VGG, the after kernel is more"global", in the sense that it is less invariant to transformations of input images that disrupt the global structure of the image while leaving the local statistics largely intact. For fully connected networks, the after kernel is less global in this sense. The after kernel tends to be more invariant to small shifts, rotations and zooms; data augmentation does not improve these invariances. The (finite approximation to the) conjugate kernel, obtained using the last layer of hidden nodes, sometimes, but not always, provides a good approximation to the NTK and the after kernel. Training a network with a larger learning rate (while holding the training error constant) produces a better kernel, as measured by the test error of a hard-margin SVM. The after kernels of networks trained with larger learning rates tend to be more global, and more invariant to small shifts, rotations and zooms.
Novel Deep Learning Architecture for Predicting Heart Disease using CNN

In the last few years, with increased population the most critical component of human life is healthcare. Compare to other deadly diseases, heart disease is one of the most lethal diseases, affecting the lives of millions of people worldwide. It is very important to detect heart disease must early so the loss of lives can be prevented. The availability of enormous amounts of data for medical diagnostics has aided in the development of complex learning-based models for automated early detection of cardiac problems. The classical machine learning approaches unable to generalize the new data sets which have not been seen in the training set. Therefore, the trained model has less accuracy in prediction stage. To minimize this issue, need to balance between training and testing datasets. This paper proposes a novel deep learning architecture using a 1D convolutional neural network for classification between healthy and non-healthy persons with balanced datasets to reduce the limitations of classical machine learning approach. Several clinical parameters are used for evaluating the risk contour in the patients which supports in early diagnosis. Various regularization methods are used to avoid overfitting in the proposed model. The proposed model achieves over 97% training accuracy and 96% test accuracy on the dataset. This is compared in detail with other machine learning algorithms using various performance parameters which proves the effectiveness of the proposed model.
Policy Mirror Descent for Regularized Reinforcement Learning: A Generalized Framework with Linear Convergence

Policy optimization, which finds the desired policy by maximizing value functions via optimization techniques, lies at the heart of reinforcement learning (RL). In addition to value maximization, other practical considerations arise as well, including the need of encouraging exploration, and that of ensuring certain structural properties of the learned policy due to safety, resource and operational constraints. These can often be accounted for via regularized RL, which augments the target value function with a structure-promoting regularizer. Focusing on discounted infinite-horizon Markov decision processes, we propose a generalized policy mirror descent (GPMD) algorithm for solving regularized RL. As a generalization of policy mirror descent (arXiv:2102.00135), our algorithm accommodates a general class of convex regularizers and promotes the use of Bregman divergence in cognizant of the regularizer in use. We demonstrate that our algorithm converges linearly to the global solution over an entire range of learning rates, in a dimension-free fashion, even when the regularizer lacks strong convexity and smoothness. In addition, this linear convergence feature is provably stable in the face of inexact policy evaluation and imperfect policy updates. Numerical experiments are provided to corroborate the appealing performance of GPMD.
Does Geopolitics Have an Impact on Energy Trade? Empirical Research on Emerging Countries

The energy trade is an important pillar of each country’s development, making up for the imbalance in the production and consumption of fossil fuels. Geopolitical risks affect the energy trade of various countries to a certain extent, but the causes of geopolitical risks are complex, and energy trade also involves many aspects, so the impact of geopolitics on energy trade is also complex. Based on the monthly data from 2000 to 2020 of 17 emerging economies, this paper employs the fixed-effect model and the regression-discontinuity (RD) model to verify the negative impact of geopolitics on energy trade first and then analyze the mechanism and heterogeneity of the impact. The following conclusions are drawn: First, geopolitics has a significant negative impact on the import and export of the energy trade, and the inhibition on the export is greater than that on the import. Second, the impact mechanism of geopolitics on the energy trade is reflected in the lagging effect and mediating effect on the imports and exports; that is, the negative impact of geopolitics on energy trade continued to be significant 10 months later. Coal and crude oil prices, as mediating variables, decreased to reduce the imports and exports, whereas natural gas prices showed an increase. Third, the impact of geopolitics on energy trade is heterogeneous in terms of national attribute characteristics and geo-event types.