Miracle-dz/UniFAR-FTUs huggingface dataset:
Nonlinear State Estimation using Gaussian Integral

In this letter, a new filtering technique to solve a nonlinear state estimation problem has been developed. It is well known that for a nonlinear system, the prior and posterior probability density functions (pdf) are non-Gaussian in nature. However, in this work, they are assumed as Gaussian and subsequently mean, and covariance of them are calculated. In the proposed method, nonlinear functions of process dynamics and measurement are expressed in a polynomial form with the help of Taylor series expansion. In order to calculate the prior and the posterior mean and covariance, the functions are integrated over the Gaussian pdf with the help of Gaussian integral. The performance of the proposed method is tested in two nonlinear state estimation problems. The simulation results show that the proposed filter provides more accurate result than other existing deterministic sample point filters such as cubature Kalman filter, unscented Kalman filter, etc.
Drug dissemination strategy with an SEIR-based SUC model

According to features of drug addiction, this paper constructs a SEIR-based SUC model to describe and predict the spread of drug addiction. Predictions are that the number of drug addictions will continue to fluctuate with a reduced amplitude and eventually stabilize. To seek the fountainhead of heroin, we identified the most likely origins of drugs in Philadelphia, PA, Cuyahoga and Hamilton, OH, Jefferson, KY, Kanawha, WV, and Bedford, VA. Based on the facts, advised concentration includes the spread of Oxycodone, Hydrocodone, Heroin and Buprenorphine. In other words, the drug transmission in the two states of Ohio and Pennsylvania require awareness. According to the propagation curve predicted by our model, the transmission of KY state is still in its early stage, while that of VA, WV is in the middle point, and OH, PA in its latter ones. Hereby the number of drug addictions in KY, OH, and VA is projected to increase in three years. For methodology, with Principal component analysis technique, 22 variables in socioeconomic data related to the continuous use of opioid drugs was filtered, where the 'Relationship' Part deserves highlight. Based on them, by using K-means algorithm, 464 counties were categorized into three basket. To combat opioid crisis, detailed action will be discussed in the sensitivity analysis section. After modeling and analytics, innovation is require to control addict and advocate anti-drug news campaigns. This part also verified the effectiveness of model when $d_1<0.2; r_1,r_2,r_3<0.3; 15<\beta_1,\beta_2,\beta_3<25$. In other words, if such boundary exceeded, number of drug addictions may rocket and peak in a short period.
Tracking the Ultraviolet Photochemistry of Thiophenone During and Beyond the Initial Ultrafast Ring Opening

Photoinduced isomerization reactions, including ring-opening reactions, lie at the heart of many processes in nature. The mechanisms of such reactions are determined by a delicate interplay of coupled electronic and nuclear dynamics unfolding on the femtosecond scale, followed by the slower redistribution of energy into different vibrational degrees of freedom. Here we apply time-resolved photoelectron spectroscopy with a seeded extreme ultraviolet free electron laser to trace the ultrafast ring opening of gas phase thiophenone molecules following photoexcitation at 265 nm. When combined with cutting edge ab initio electronic structure and molecular dynamics calculations of both the excited and ground state molecules, the results provide unprecedented insights into both electronic and nuclear dynamics of this fundamental class of reactions. The initial ring opening and non-adiabatic coupling to the electronic ground state is shown to be driven by ballistic SC bond extension and to be complete within 350 femtoseconds. Theory and experiment also allow clear visualization of the rich ground-state dynamics involving formation of, and interconversion between, several ring opened isomers and the reformed cyclic structure, and fragmentation (CO loss) over much longer timescales.
Transferable Force-Torque Dynamics Model for Peg-in-hole Task

We present a learning-based force-torque dynamics to achieve model-based control for contact-rich peg-in-hole task using force-only inputs. Learning the force-torque dynamics is challenging because of the ambiguity of the low-dimensional 6-d force signal and the requirement of excessive training data. To tackle these problems, we propose a multi-pose force-torque state representation, based on which a dynamics model is learned with the data generated in a sample-efficient offline fashion. In addition, by training the dynamics model with peg-and-holes of various shapes, scales, and elasticities, the model could quickly transfer to new peg-and-holes after a small number of trials. Extensive experiments show that our dynamics model could adapt to unseen peg-and-holes with 70% fewer samples required compared to learning from scratch. Along with the learned dynamics, model predictive control and model-based reinforcement learning policies achieve over 80% insertion success rate. Our video is available at this https URL.
Training the convolutional neural network with statistical dependence of the response on the input data distortion

The paper proposes an approach to training a convolutional neural network using information on the level of distortion of input data. The learning process is modified with an additional layer, which is subsequently deleted, so the architecture of the original network does not change. As an example, the LeNet5 architecture network with training data based on the MNIST symbols and a distortion model as Gaussian blur with a variable level of distortion is considered. This approach does not have quality loss of the network and has a significant error-free zone in responses on the test data which is absent in the traditional approach to training. The responses are statistically dependent on the level of input image’s distortions and there is a presence of a strong relationship between them.
Mis-classified Vector Guided Softmax Loss for Face Recognition

Face recognition has witnessed significant progress due to the advances of deep convolutional neural networks (CNNs), the central task of which is how to improve the feature discrimination. To this end, several margin-based (e.g., angular, additive and additive angular margins) softmax loss functions have been proposed to increase the feature margin between different classes. However, despite great achievements have been made, they mainly suffer from three issues: 1) Obviously, they ignore the importance of informative features mining for discriminative learning; 2) They encourage the feature margin only from the ground truth class, without realizing the discriminability from other non-ground truth classes; 3) The feature margin between different classes is set to be same and fixed, which may not adapt the situations very well. To cope with these issues, this paper develops a novel loss function, which adaptively emphasizes the mis-classified feature vectors to guide the discriminative feature learning. Thus we can address all the above issues and achieve more discriminative face features. To the best of our knowledge, this is the first attempt to inherit the advantages of feature margin and feature mining into a unified loss function. Experimental results on several benchmarks have demonstrated the effectiveness of our method over state-of-the-art alternatives. Our code is available at http://www.cbsr. ia.ac. cn/users/xiaobowang/.
An Energy Efficient Distributed Gossip Algorithm for Wireless Sensor Networks based on a Randomized Markovian Duty-Cycling

This paper proposes a novel asynchronous consensus algorithm which is based on a continuous update rule and an energy efficient event triggered duty (wake-sleep) cycle based on a discrete Markov chain model. The system model of the proposed algorithm is formulated and analyzed. The conditions for convergence and stability of the algorithm are derived and the algorithm is proved to converge to an average consensus. Numerical simulations on random, circular, chain and star graphs show stability in consensus and convergence to network global state average.
Stability of Redundancy Systems with Processor Sharing

We investigate the stability condition for redundancy-d systems where each of the servers follows a processor-sharing (PS) discipline. We allow for generally distributed job sizes, with possible dependence among the d replica sizes being governed by an arbitrary joint distribution. We establish that the stability condition for the associated fluid-limit model is characterized by the expectation of the minimum of d replica sizes being less than the mean interarrival time per server. In the special case of identical replicas, the stability condition is insensitive to the job size distribution given its mean, and the stability condition is inversely proportional to the number of replicas. In the special case of i.i. d. replicas, the stability threshold decreases (increases) in the number of replicas for job size distributions that are NBU (NWU). We also discuss extensions to scenarios with heterogeneous servers.
Capacity-Approaching Polar Codes With Long Codewords and Successive Cancellation Decoding Based on Improved Gaussian Approximation

This paper focuses on an improved Gaussian approximation (GA) based construction of polar codes with successive cancellation (SC) decoding over an additive white Gaussian noise (AWGN) channel. Arıkan proved that polar codes with low-complexity SC decoding can approach the channel capacity of an arbitrary symmetric binary-input discrete memoryless channel, provided that the code length is chosen large enough. Nevertheless, how to construct such codes over an AWGN channel with low computational effort has been an open problem. Compared to density evolution, the GA is known as a low complexity yet powerful technique that traces the evolution of the mean log likelihood ratio (LLR) value by iterating a nonlinear function. Therefore, its high-precision numerical evaluation is critical as the code length increases. In this work, by analyzing the asymptotic behavior of this nonlinear function, we propose an improved GA approach that makes an accurate trace of mean LLR evolution feasible. With this improved GA, through numerical analysis and simulations with code lengths up to $N=2^{18}$ , we explicitly demonstrate that various code-rate polar codes with long codeword and capacity approaching behavior can be easily designed.
On the geometry of Stein variational gradient descent

Bayesian inference problems require sampling or approximating high-dimensional probability distributions. The focus of this paper is on the recently introduced Stein variational gradient descent methodology, a class of algorithms that rely on iterated steepest descent steps with respect to a reproducing kernel Hilbert space norm. This construction leads to interacting particle systems, the mean-field limit of which is a gradient flow on the space of probability distributions equipped with a certain geometrical structure. We leverage this viewpoint to shed some light on the convergence properties of the algorithm, in particular addressing the problem of choosing a suitable positive definite kernel function. Our analysis leads us to considering certain nondifferentiable kernels with adjusted tails. We demonstrate significant performs gains of these in various numerical experiments.
Artificial boundary method for the solution of pricing European options under the Heston model

This paper considers the valuation of a European call option under the Heston stochastic volatility model. We present the asymptotic solution to the option pricing problem in powers of the volatility of variance. Then we introduce the artificial boundary method for solving the problem on a truncated domain, and derive several artificial boundary conditions (ABCs) on the artificial boundary of the bounded computational domain. A typical finite difference scheme and quadrature rule are used for the numerical solution of the reduced problem. Numerical experiments show that the proposed ABCs are able to improve the accuracy of the results and have a significant advantage over the widely-used boundary conditions by Heston in the original paper (Heston, 1993).
Active Search for Nearest Neighbors

In pattern recognition or machine learning, it is a very fundamental task to find nearest neighbors of a given point. All the methods for the task work basically by comparing the given point to all the points in the data set. That is why the computational cost increases with the number of data points. However, the human visual system seems to work in a different way. When the human visual system tries to find the neighbors of one point on a map, it directly focuses on the area around the point and actively searches the neighbors by looking or zooming in and out around the point. In this paper, we propose an innovative search method for nearest neighbors, which seems very similar to how human visual system works on the task.
MANELA: A Multi-Agent Algorithm for Learning Network Embeddings

Playing an essential role in data mining, machine learning has a long history of being applied to networks on multifarious tasks and has played an essential role in data mining. However, the discrete and sparse natures of networks often render it difficult to apply machine learning directly to networks. To circumvent this difficulty, one major school of thought to approach networks using machine learning is via network embeddings. On the one hand, this network embeddings have achieved huge success on aggregated network data in recent years. On the other hand, learning network embeddings on distributively stored networks still remained understudied: To the best of our knowledge, all existing algorithms for learning network embeddings have hitherto been exclusively centralized and thus cannot be applied to these networks. To accommodate distributively stored networks, in this paper, we proposed a multi-agent model. Under this model, we developed the multi-agent network embedding learning algorithm (MANELA) for learning network embeddings. We demonstrate MANELA's advantages over other existing centralized network embedding learning algorithms both theoretically and experimentally. Finally, we further our understanding in MANELA via visualization and exploration of its relationship to DeepWalk.
Sparse Graph Attention Networks

Graph Neural Networks (GNNs) have proved to be an effective representation learning framework for graph-structured data, and have achieved state-of-the-art performance on many practical predictive tasks, such as node classification, link prediction and graph classification. Among the variants of GNNs, Graph Attention Networks (GATs) learn to assign dense attention coefficients over all neighbors of a node for feature aggregation, and improve the performance of many graph learning tasks. However, real-world graphs are often very large and noisy, and GATs are prone to overfitting if not regularized properly. Even worse, the local aggregation mechanism of GATs may fail on disassortative graphs, where nodes within local neighborhood provide more noise than useful information for feature aggregation. In this paper, we propose Sparse Graph Attention Networks (SGATs) that learn sparse attention coefficients under an <inline-formula><tex-math notation="LaTeX">$L_0$</tex-math><alternatives><mml:math><mml:msub><mml:mi>L</mml:mi><mml:mn>0</mml:mn></mml:msub></mml:math><inline-graphic xlink:href="ji-ieq1-3072345.gif"/></alternatives></inline-formula>-norm regularization, and the learned sparse attentions are then used for all GNN layers, resulting in an edge-sparsified graph. By doing so, we can identify noisy/task-irrelevant edges, and thus perform feature aggregation on most informative neighbors. Extensive experiments on synthetic and real-world (assortative and disassortative) graph learning benchmarks demonstrate the superior performance of SGATs. In particular, SGATs can remove about 50-80 percent edges from large assortative graphs, such as PPI and Reddit, while retaining similar classification accuracies. On disassortative graphs, SGATs prune majority of noisy edges and outperform GATs in classification accuracies by significant margins. Furthermore, the removed edges can be interpreted intuitively and quantitatively. To the best of our knowledge, this is the first graph learning algorithm that shows significant redundancies in graphs and edge-sparsified graphs can achieve similar (on assortative graphs) or sometimes higher (on disassortative graphs) predictive performances than original graphs. Our code is available at <uri>https://github.com/Yangyeeee/SGAT</uri>.
Generalizable prediction of academic performance from short texts on social media

It has already been established that digital traces can be used to predict various human attributes. In most cases, however, predictive models rely on features that are specific to a particular source of digital trace data. In contrast, short texts written by users $-$ tweets, posts, or comments $-$ are ubiquitous across multiple platforms. In this paper, we explore the predictive power of short texts with respect to the academic performance of their authors. We use data from a representative panel of Russian students that includes information about their educational outcomes and activity on a popular networking site, VK. We build a model to predict academic performance from users' posts on VK and then apply it to a different context. In particular, we show that the model could reproduce rankings of schools and universities from the posts of their students on social media. We also find that the same model could predict academic performance from tweets as well as from VK posts. The generalizability of a model trained on a relatively small data set could be explained by the use of continuous word representations trained on a much larger corpus of social media posts. This also allows for greater interpretability of model predictions.
The structure and stability of extended, inclined circumplanetary disc or ring systems

Large dips in the brightness for a number of stars have been observed, for which the tentative explanation is occultation of the star by a transiting circumplanetary disk or ring system. In order for the circumplanetary disk/rings to block the host star's light, the disk must be tilted out of the planet's orbital plane, which poses stability problems due to the radial extent of the disk required to explain the brightness dip durations. This work uses N-body integrations to study the structure and stability of circumplanetary disk/ring systems tilted out of the planet's orbital plane by the spinning planet's mass quadrupole. Simulating the disk as a collection of test particles with orbits initialized near the Laplace surface (equilibrium between tidal force from host star and force from planet's mass quadrupole), we find that many extended, inclined circumplanetary disks remain stable over the duration of the integrations (~3-16 Myr). Two dynamical resonances/instabilities excite the particle eccentricities and inclinations: the Lidov-Kozai effect which occurs in the disk's outer regions, and ivection resonance which occurs in the disk's inner regions. Our work places constraints on the maximum radial extent of inclined circumplanetary disk/ring systems, and shows that gaps present in circumplanetary disks do not necessarily imply the presence of exomoons.
Image Reconstruction via Discrete Curvatures

The curvature regularities are well-known for providing strong priors in the continuity of edges, which have been applied to a wide range of applications in image processing and computer vision. However, these models are usually non-convex, non-smooth and highly non-linear, the first-order optimal condition of which are high-order partial differential equations. Thus, the numerical computation are extremely challenging. In this paper, we propose to estimate the discrete curvatures, i.e., mean curvature and Gaussian curvature, in the local neighborhood according to differential geometry theory. By minimizing certain functions of curvatures on all level curves of an image, it yields a kind of weighted total variation minimization problem, which can be efficiently solved by the proximal alternating direction method of multipliers (ADMM). Numerical experiments are implemented to demonstrate the effectiveness and superiority of our proposed variational models for different image reconstruction tasks.
BLiMP: The Benchmark of Linguistic Minimal Pairs for English

Abstract We introduce The Benchmark of Linguistic Minimal Pairs (BLiMP),1 a challenge set for evaluating the linguistic knowledge of language models (LMs) on major grammatical phenomena in English. BLiMP consists of 67 individual datasets, each containing 1,000 minimal pairs—that is, pairs of minimally different sentences that contrast in grammatical acceptability and isolate specific phenomenon in syntax, morphology, or semantics. We generate the data according to linguist-crafted grammar templates, and human aggregate agreement with the labels is 96.4%. We evaluate n-gram, LSTM, and Transformer (GPT-2 and Transformer-XL) LMs by observing whether they assign a higher probability to the acceptable sentence in each minimal pair. We find that state-of-the-art models identify morphological contrasts related to agreement reliably, but they struggle with some subtle semantic and syntactic phenomena, such as negative polarity items and extraction islands.
Improving Question Generation with Sentence-level Semantic Matching and Answer Position Inferring

Taking an answer and its context as input, sequence-to-sequence models have made considerable progress on question generation. However, we observe that these approaches often generate wrong question words or keywords and copy answer-irrelevant words from the input. We believe that lacking global question semantics and exploiting answer position-awareness not well are the key root causes. In this paper, we propose a neural question generation model with two general modules: sentence-level semantic matching and answer position inferring. Further, we enhance the initial state of the decoder by leveraging the answer-aware gated fusion mechanism. Experimental results demonstrate that our model outperforms the state-of-the-art (SOTA) models on SQuAD and MARCO datasets. Owing to its generality, our work also improves the existing models significantly.
Machine learning applications in time series hierarchical forecasting

Hierarchical forecasting (HF) is needed in many situations in the supply chain (SC) because managers often need different levels of forecasts at different levels of SC to make a decision. Top-Down (TD), Bottom-Up (BU) and Optimal Combination (COM) are common HF models. These approaches are static and often ignore the dynamics of the series while disaggregating them. Consequently, they may fail to perform well if the investigated group of time series are subject to large changes such as during the periods of promotional sales. We address the HF problem of predicting real-world sales time series that are highly impacted by promotion. We use three machine learning (ML) models to capture sales variations over time. Artificial neural networks (ANN), extreme gradient boosting (XGboost), and support vector regression (SVR) algorithms are used to estimate the proportions of lower-level time series from the upper level. We perform an in-depth analysis of 61 groups of time series with different volatilities and show that ML models are competitive and outperform some well-established models in the literature.