pwc-archive/papers-with-abstracts huggingface dataset:
FAST: Improving Controllability for Text Generation with Feedback Aware Self-Training
Authors: Junyi Chai, Reid Pryzant, Victor Ye Dong, Konstantin Golobokov, Chenguang Zhu, Yi Liu
Date: 2022-10-06 00:00:00
Tasks: Attribute, Causal Inference, Conditional Text Generation, counterfactual, Text Generation

Controllable text generation systems often leverage control codes to direct various properties of the output like style and length. Inspired by recent work on causal inference for NLP, this paper reveals a previously overlooked flaw in these control code-based conditional text generation algorithms. Spurious correlations in the training data can lead models to incorrectly rely on parts of the input other than the control code for attribute selection, significantly undermining downstream generation quality and controllability. We demonstrate the severity of this issue with a series of case studies and then propose two simple techniques to reduce these correlations in training sets. The first technique is based on resampling the data according to an example's propensity towards each linguistic attribute (IPS). The second produces multiple counterfactual versions of each example and then uses an additional feedback mechanism to remove noisy examples (feedback aware self-training, FAST). We evaluate on 3 tasks -- news headline, meta review, and search ads generation -- and demonstrate that FAST can significantly improve the controllability and language quality of generated outputs when compared to state-of-the-art controllable text generation approaches.
A Step Towards Uncovering The Structure of Multistable Neural Networks
Authors: Magnus Tournoy, Brent Doiron
Date: 2022-10-06 00:00:00

We study how the connectivity within a recurrent neural network determines and is determined by the multistable solutions of network activity. To gain analytic tractability we let neural activation be a non-smooth Heaviside step function. This nonlinearity partitions the phase space into regions with different, yet linear dynamics. In each region either a stable equilibrium state exists, or network activity flows to outside of the region. The stable states are identified by their semipositivity constraints on the synaptic weight matrix. The restrictions can be separated by their effects on the signs or the strengths of the connections. Exact results on network topology, sign stability, weight matrix factorization, pattern completion and pattern coupling are derived and proven. Our work may lay the foundation for multistability in more complex recurrent neural networks.
Genetic algorithm formulation and tuning with use of test functions
Authors: Tomasz Tarkowski
Date: 2022-10-06 00:00:00

This work discusses single-objective constrained genetic algorithm with floating-point, integer, binary and permutation representation. Floating-point genetic algorithm tuning with use of test functions is done and leads to a parameterization with comparatively outstanding performance.
KAST: Knowledge Aware Adaptive Session Multi-Topic Network for Click-Through Rate Prediction
Authors: Dike Sun, Kai Liu, ShengKai Yang
Date: 2022-10-07 00:00:00
Tasks: Click-Through Rate Prediction, Recommendation Systems

Capturing the evolving trends of user interest is important for both recommendation systems and advertising systems, and user behavior sequences have been successfully used in Click-Through-Rate(CTR) prediction problems. However, if the user interest is learned on the basis of item-level behaviors, the performance may be affected by the following two issues. Firstly, some casual outliers might be included in the behavior sequences as user behaviors are likely to be diverse. Secondly, the span of time intervals between user behaviors is random and irregular, for which a RNN-based module employed from NLP is not perfectly adaptive. To handle these two issues, we propose the Knowledge aware Adaptive Session multi-Topic network(KAST). It can adaptively segment user sessions from the whole user behavior sequence, and maintain similar intents in the same session. Furthermore, in order to improve the quality of session segmentation and representation, a knowledge-aware module is introduced so that the structural information from the user-item interaction can be extracted in an end-to-end manner, and a marginal based loss with these information is merged into the major loss. Through extensive experiments on public benchmarks, we demonstrate that KAST can achieve superior performance than state-of-the-art methods for CTR prediction, and key modules and hyper-parameters are also evaluated.
High Resolution Spatio-Temporal Model for Room-Level Airborne Pandemic Spread
Authors: Teddy Lazebnik, Ariel Alexi
Date: 2022-10-07 00:00:00
Tasks: Vocal Bursts Intensity Prediction

Airborne pandemics have caused millions of deaths worldwide, large-scale economic losses, and catastrophic sociological shifts in human history. Researchers have developed multiple mathematical models and computational frameworks to investigate and predict the pandemic spread on various levels and scales such as countries, cities, large social events, and even buildings. However, modeling attempts of airborne pandemic dynamics on the smallest scale, a single room, have been mostly neglected. As time indoors increases due to global urbanization processes, more infections occur in shared rooms. In this study, a high-resolution spatio-temporal epidemiological model with airflow dynamics to evaluate airborne pandemic spread is proposed. The model is implemented using high-resolution 3D data obtained using a light detection and ranging (LiDAR) device and computing the model based on the Computational Fluid Dynamics (CFD) model for the airflow and the Susceptible-Exposed-Infected (SEI) model for the epidemiological dynamics. The pandemic spread is evaluated in four types of rooms, showing significant differences even for a short exposure duration. We show that the room's topology and individual distribution in the room define the ability of air ventilation to reduce pandemic spread throughout breathing zone infection.
Scientific and Technological News Recommendation Based on Knowledge Graph with User Perception
Authors: Yuyao Zeng, Junping Du, Zhe Xue, Ang Li
Date: 2022-10-07 00:00:00
Tasks: Collaborative Filtering, News Recommendation, Recommendation Systems, Relation

Existing research usually utilizes side information such as social network or item attributes to improve the performance of collaborative filtering-based recommender systems. In this paper, the knowledge graph with user perception is used to acquire the source of side information. We proposed KGUPN to address the limitations of existing embedding-based and path-based knowledge graph-aware recommendation methods, an end-to-end framework that integrates knowledge graph and user awareness into scientific and technological news recommendation systems. KGUPN contains three main layers, which are the propagation representation layer, the contextual information layer and collaborative relation layer. The propagation representation layer improves the representation of an entity by recursively propagating embeddings from its neighbors (which can be users, news, or relationships) in the knowledge graph. The contextual information layer improves the representation of entities by encoding the behavioral information of entities appearing in the news. The collaborative relation layer complements the relationship between entities in the news knowledge graph. Experimental results on real-world datasets show that KGUPN significantly outperforms state-of-the-art baselines in scientific and technological news recommendation.
Unsupervised Semantic Representation Learning of Scientific Literature Based on Graph Attention Mechanism and Maximum Mutual Information
Authors: Hongrui Gao, Yawen Li, Meiyu Liang, Zeli Guan
Date: 2022-10-07 00:00:00
Tasks: Contrastive Learning, Graph Attention, Graph Neural Network, Node Classification, Representation Learning

Since most scientific literature data are unlabeled, this makes unsupervised graph-based semantic representation learning crucial. Therefore, an unsupervised semantic representation learning method of scientific literature based on graph attention mechanism and maximum mutual information (GAMMI) is proposed. By introducing a graph attention mechanism, the weighted summation of nearby node features make the weights of adjacent node features entirely depend on the node features. Depending on the features of the nearby nodes, different weights can be applied to each node in the graph. Therefore, the correlations between vertex features can be better integrated into the model. In addition, an unsupervised graph contrastive learning strategy is proposed to solve the problem of being unlabeled and scalable on large-scale graphs. By comparing the mutual information between the positive and negative local node representations on the latent space and the global graph representation, the graph neural network can capture both local and global information. Experimental results demonstrate competitive performance on various node classification benchmarks, achieving good results and sometimes even surpassing the performance of supervised learning.
A Relational Triple Extraction Method Based on Feature Reasoning for Technological Patents
Authors: Runze Fang, Junping Du, Yingxia Shao, Zeli Guan
Date: 2022-10-07 00:00:00
Tasks: Relation

The relation triples extraction method based on table filling can address the issues of relation overlap and bias propagation. However, most of them only establish separate table features for each relationship, which ignores the implicit relationship between different entity pairs and different relationship features. Therefore, a feature reasoning relational triple extraction method based on table filling for technological patents is proposed to explore the integration of entity recognition and entity relationship, and to extract entity relationship triples from multi-source scientific and technological patents data. Compared with the previous methods, the method we proposed for relational triple extraction has the following advantages: 1) The table filling method that saves more running space enhances the speed and efficiency of the model. 2) Based on the features of existing token pairs and table relations, reasoning the implicit relationship features, and improve the accuracy of triple extraction. On five benchmark datasets, we evaluated the model we suggested. The result suggest that our model is advanced and effective, and it performed well on most of these datasets.
Embedding Representation of Academic Heterogeneous Information Networks Based on Federated Learning
Authors: Junfu Wang, Yawen Li, Meiyu Liang, Ang Li
Date: 2022-10-07 00:00:00
Tasks: Blocking, Federated Learning, Network Embedding, Representation Learning

Academic networks in the real world can usually be portrayed as heterogeneous information networks (HINs) with multi-type, universally connected nodes and multi-relationships. Some existing studies for the representation learning of homogeneous information networks cannot be applicable to heterogeneous information networks because of the lack of ability to issue heterogeneity. At the same time, data has become a factor of production, playing an increasingly important role. Due to the closeness and blocking of businesses among different enterprises, there is a serious phenomenon of data islands. To solve the above challenges, aiming at the data information of scientific research teams closely related to science and technology, we proposed an academic heterogeneous information network embedding representation learning method based on federated learning (FedAHE), which utilizes node attention and meta path attention mechanism to learn low-dimensional, dense and real-valued vector representations while preserving the rich topological information and meta-path-based semantic information of nodes in network. Moreover, we combined federated learning with the representation learning of HINs composed of scientific research teams and put forward a federal training mechanism based on dynamic weighted aggregation of parameters (FedDWA) to optimize the node embeddings of HINs. Through sufficient experiments, the efficiency, accuracy and feasibility of our proposed framework are demonstrated.
Scientific Paper Classification Based on Graph Neural Network with Hypergraph Self-attention Mechanism
Authors: Jiashun Liu, Zhe Xue, Ang Li
Date: 2022-10-07 00:00:00
Tasks: Graph Neural Network, Management

The number of scientific papers has increased rapidly in recent years. How to make good use of scientific papers for research is very important. Through the high-quality classification of scientific papers, researchers can quickly find the resource content they need from the massive scientific resources. The classification of scientific papers will effectively help researchers filter redundant information, obtain search results quickly and accurately, and improve the search quality, which is necessary for scientific resource management. This paper proposed a science-technique paper classification method based on hypergraph neural network(SPHNN). In the heterogeneous information network of scientific papers, the repeated high-order subgraphs are modeled as hyperedges composed of multiple related nodes. Then the whole heterogeneous information network is transformed into a hypergraph composed of different hyperedges. The graph convolution operation is carried out on the hypergraph structure, and the hyperedges self-attention mechanism is introduced to aggregate different types of nodes in the hypergraph, so that the final node representation can effectively maintain high-order nearest neighbor relationships and complex semantic information. Finally, by comparing with other methods, we proved that the model proposed in this paper has improved its performance.
OQM9HK: A Large-Scale Graph Dataset for Machine Learning in Materials Science
Authors: Takenori Yamamoto
Date: 2022-09-30 00:00:00
Proceeding: Technical report, RIMCS LLC 2022 9
Tasks: Band Gap, Formation Energy, Graph Representation Learning, Property Prediction, Representation Learning, Total Magnetization

We introduce a large-scale dataset of quantum-mechanically calculated properties of crystalline materials for graph representation learning that contains approximately 900k entries (OQM9HK). This dataset is constructed on the basis of the Open Quantum Materials Database (OQMD) v1.5 containing more than one million entries, and is the successor to the OQMD v1.2 dataset containing approximately 600k entries (OQM6HK). We develop the graph creation algorithm to produce a binary edge-labeled (BEL) graph representing a crystalline material. The BEL graph has higher representability of crystal structure than the edge-unlabeled ones. In materials property prediction tasks, crystal graph neural networks trained on the BEL graph dataset perform better than ones on the other graph datasets. The OQM9HK graph dataset is available at the Zenodo repository, https://doi.org/10.5281/zenodo.7124330
Mlphon: A Multifunctional Grapheme-Phoneme Conversion Tool Using Finite State Transducers
Authors: Kavya Manohar, A R jayan, Rajeev Rajan
Date: 2022-09-05 00:00:00
Proceeding: IEEE Access 2022 9
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Diversity, Grapheme-to-Phoneme Conversion, Language Modeling, Language Modelling, Phoneme Recognition, Pronunciation Dictionary Creation, speech-recognition, Speech Recognition, Speech Synthesis, text-to-speech, Text to Speech, Text-To-Speech Synthesis

In this article we present the design and the development of a knowledge based computational linguistic tool, Mlphon for Malayalam language. Mlphon computationally models linguistic rules using finite state transducers and performs multiple functions including grapheme to phoneme (g2p) and phoneme to grapheme (p2g) conversions, syllabification, phonetic feature analysis and script grammar check. This open source software tool, released under MIT license, is developed as a one-stop solution to handle different speech related text processing tasks for automatic speech recognition, text to speech synthesis and non-speech natural language processing tasks including syllable subword based language modeling, phoneme diversity analysis and text sanity check. The tool is evaluated on a manually crafted gold standard lexicon. Mlphon performs orthographic syllabification with 99% accuracy with a syllable error rate of 0.62% on the gold standard lexicon. For grapheme to phoneme conversion task, overall phoneme recognition accuracy of 99% with a phoneme error rate of 0.55% is obtained on gold standard lexicon. Additionally an extrinsic evaluation of Mlphon is performed by employing the pronunciation lexicon created using Mlphon, in Malayalam automatic speech recognition (ASR) task. Performance analysis in terms of the computation time of lexicon creation process and the word error rate (WER) on ASR task are presented along with a comparison over other automated tools for lexicon creation. Pronunciation lexicons with more than 100k commonly used Malayalam words in phonemised and syllabified forms is created and they are published as open language resources along with this work. We also demonstrate the usage of Mlphon on different natural language processing applications - syllable subword ASR, assisted pronunciation learning, phoneme diversity analysis and text sanity check. Being a knowledge based solution with open source code, Mlphon can be adapted to other languages of similar script nature.
Sparse model selection via integral terms
Authors: Hayden Schaeffer, Scott G. McCalla
Date: 2017-08-02 00:00:00
Proceeding: PHYSICAL REVIEW E 96 2017 8
Tasks: model, Model Selection, parameter estimation, regression

Model selection and parameter estimation are important for the effective integration of experimental data,
scientific theory, and precise simulations. In this work, we develop a learning approach for the selection and
identification of a dynamical system directly from noisy data. The learning is performed by extracting a small
subset of important features from an overdetermined set of possible features using a nonconvex sparse regression
model. The sparse regression model is constructed to fit the noisy data to the trajectory of the dynamical
system while using the smallest number of active terms. Computational experiments detail the model’s stability,
robustness to noise, and recovery accuracy. Examples include nonlinear equations, population dynamics, chaotic
systems, and fast-slow systems.
Maximum Likelihood-based Gridless DoA Estimation Using Structured Covariance Matrix Recovery and SBL with Grid Refinement
Authors: Rohan R. Pote, Bhaskar D. Rao
Date: 2022-10-07 00:00:00
Tasks: Direction of Arrival Estimation

We consider the parametric data model employed in applications such as line spectral estimation and direction-of-arrival estimation. We focus on the stochastic maximum likelihood estimation (MLE) framework and offer approaches to estimate the parameter of interest in a gridless manner, overcoming the model complexities of the past. This progress is enabled by the modern trend of reparameterization of the objective and exploiting the sparse Bayesian learning (SBL) approach. The latter is shown to be a correlation-aware method, and for the underlying problem it is identified as a grid-based technique for recovering a structured covariance matrix of the measurements. For the case when the structured matrix is expressible as a sampled Toeplitz matrix, such as when measurements are sampled in time or space at regular intervals, additional constraints and reparameterization of the SBL objective leads to the proposed structured matrix recovery technique based on MLE. The proposed optimization problem is non-convex, and we propose a majorization-minimization based iterative procedure to estimate the structured matrix; each iteration solves a semidefinite program. We recover the parameter of interest in a gridless manner by appealing to the Caratheodory-Fejer result on decomposition of PSD Toeplitz matrices. For the general case of irregularly spaced time or spatial samples, we propose an iterative SBL procedure that refines grid points to increase resolution near potential source locations, while maintaining a low per iteration complexity. We provide numerical results to evaluate and compare the performance of the proposed techniques with other gridless techniques, and the CRB. The proposed correlation-aware approach is more robust to environmental/system effects such as low number of snapshots, correlated sources, small separation between source locations and improves sources identifiability.
On-line Motion Planning Using Bernstein Polynomials for Enhanced Target Localization in Autonomous Vehicles
Authors: Camilla Tabasso, Venanzio Cichella
Date: 2022-10-06 00:00:00
Tasks: Autonomous Vehicles, Motion Planning, Position

The use of autonomous vehicles for target localization in modern applications has emphasized their superior efficiency, improved safety, and cost advantages over human-operated methods. For localization tasks, autonomous vehicles can be used to increase efficiency and ensure that the target is localized as quickly and precisely as possible. However, devising a motion planning scheme to achieve these objectives in a computationally efficient manner suitable for real-time implementation is not straightforward. In this paper, we introduce a motion planning solution for enhanced target localization, leveraging Bernstein polynomial basis functions to approximate the probability distribution of the target's trajectory. This allows us to derive estimation performance criteria which are used by the motion planner to enhance the estimator efficacy. To conclude, we present simulation results that validate the effectiveness of the suggested algorithm.
Inertia constants for individual power plants
Authors: David Kraljic, Blaz Sobocan, Jernej Katanec, Matej Logar, Miha Troha
Date: 2022-10-07 00:00:00

Keeping the power system stable is becoming more challenging with the growing share of renewable energy sources of low or negligible inertia. Inertia constants for individual power plants are generally not known and are roughly estimated by considering the type of power generation technology and the nameplate capacity of plants. More accurate knowledge of inertia constants of individual power plants would give greater transparency to decisions of the transmission system operator (TSO) and to auctions for the procurement of inertia services. Additionally, a more accurate forecast or estimation of system inertia would improve the price signals to the power market well in advance of balancing actions taken by the TSO. We develop methods based on a combination of mathematical optimisation and machine learning that reverse-engineer the inertia constants of individual power plants from the historical values of their power production and from aggregate values of inertia in a power system. We demonstrate the methods for the power system of Great Britain (GB), where historical values for aggregate inertia are published by the TSO. We show that the recovered inertia data is crucial in understanding certain individual balancing decisions by the TSO. We use the reverse-engineered inertia constants to predict system inertia which gives valuable information to the power market.