pwc-archive/papers-with-abstracts huggingface dataset:
RoBERTuito: a pre-trained language model for social media text in Spanish
Authors: Juan Manuel Pérez, Damián A. Furman, Laura Alonso Alemany, Franco Luque
Date: 2021-11-18 00:00:00
Proceeding: LREC 2022 6
Tasks: Language Modeling, Language Modelling, Natural Language Understanding, Transfer Learning

Since BERT appeared, Transformer language models and transfer learning have become state-of-the-art for Natural Language Understanding tasks. Recently, some works geared towards pre-training specially-crafted models for particular domains, such as scientific papers, medical documents, user-generated texts, among others. These domain-specific models have been shown to improve performance significantly in most tasks. However, for languages other than English such models are not widely available. In this work, we present RoBERTuito, a pre-trained language model for user-generated text in Spanish, trained on over 500 million tweets. Experiments on a benchmark of tasks involving user-generated text showed that RoBERTuito outperformed other pre-trained language models in Spanish. In addition to this, our model achieves top results for some English-Spanish tasks of the Linguistic Code-Switching Evaluation benchmark (LinCE) and has also competitive performance against monolingual models in English tasks. To facilitate further research, we make RoBERTuito publicly available at the HuggingFace model hub together with the dataset used to pre-train it.
Continuous learning of spiking networks trained with local rules
Authors: Dmitry Antonov, Kirill Sviatov, Sergey Sukhov
Date: 2021-11-18 00:00:00

Artificial neural networks (ANNs) experience catastrophic forgetting (CF) during sequential learning. In contrast, the brain can learn continuously without any signs of catastrophic forgetting. Spiking neural networks (SNNs) are the next generation of ANNs with many features borrowed from biological neural networks. Thus, SNNs potentially promise better resilience to CF. In this paper, we study the susceptibility of SNNs to CF and test several biologically inspired methods for mitigating catastrophic forgetting. SNNs are trained with biologically plausible local training rules based on spike-timing-dependent plasticity (STDP). Local training prohibits the direct use of CF prevention methods based on gradients of a global loss function. We developed and tested the method to determine the importance of synapses (weights) based on stochastic Langevin dynamics without the need for the gradients. Several other methods of catastrophic forgetting prevention adapted from analog neural networks were tested as well. The experiments were performed on freely available datasets in the SpykeTorch environment.
Evolutionary Algorithm for Graph Coloring Problem
Authors: Robiul Islam, Arup Kumar Pramanik
Date: 2021-11-17 00:00:00

The graph coloring problem (GCP) is one of the most studied NP-HARD problems in computer science. Given a graph , the task is to assign a color to all vertices such that no vertices sharing an edge receive the same color and that the number of used colors, is minimal. Different heuristic, meta-heuristic, machine learning and hybrid solution methods have been applied to obtain the solution. To solve this problem we use mutation of evolutionary algorithm. For this purpose we introduce binary encoding for Graph Coloring Problem. This binary encoding help us for mutation, evaluate, immune system and merge color easily and also reduce coloring dynamically. In the traditional evolutionary algorithm (EA) for graph coloring, k-coloring approach is used and the EA is run repeatedly until the lowest possible is reached. In our paper, we start with the theoretical upper bound of chromatic number, that is, maximum out-degree + 1 and in the process of evolution some of the colors are made unused to dynamically reduce the number of color in every generation. We test few standard DIMACS benchmark and compare resent paper. Maximum results are same as expected chromatic color and few data sets are larger than expected chromatic number
Locally Learned Synaptic Dropout for Complete Bayesian Inference
Authors: Kevin L. McKee, Ian C. Crandell, Rishidev Chaudhuri, Randall C. O'Reilly
Date: 2021-11-18 00:00:00
Tasks: Bayesian Inference

The Bayesian brain hypothesis postulates that the brain accurately operates on statistical distributions according to Bayes' theorem. The random failure of presynaptic vesicles to release neurotransmitters may allow the brain to sample from posterior distributions of network parameters, interpreted as epistemic uncertainty. It has not been shown previously how random failures might allow networks to sample from observed distributions, also known as aleatoric or residual uncertainty. Sampling from both distributions enables probabilistic inference, efficient search, and creative or generative problem solving. We demonstrate that under a population-code based interpretation of neural activity, both types of distribution can be represented and sampled with synaptic failure alone. We first define a biologically constrained neural network and sampling scheme based on synaptic failure and lateral inhibition. Within this framework, we derive drop-out based epistemic uncertainty, then prove an analytic mapping from synaptic efficacy to release probability that allows networks to sample from arbitrary, learned distributions represented by a receiving layer. Second, our result leads to a local learning rule by which synapses adapt their release probabilities. Our result demonstrates complete Bayesian inference, related to the variational learning method of dropout, in a biologically constrained network using only locally-learned synaptic failure rates.
Learning Ultrasound Scanning Skills from Human Demonstrations
Authors: Xutian Deng, Ziwei Lei, Yi Wang, Miao Li
Date: 2021-11-09 00:00:00

Recently, the robotic ultrasound system has become an emerging topic owing to the widespread use of medical ultrasound. However, it is still a challenging task to model and to transfer the ultrasound skill from an ultrasound physician. In this paper, we propose a learning-based framework to acquire ultrasound scanning skills from human demonstrations. First, the ultrasound scanning skills are encapsulated into a high-dimensional multi-modal model in terms of interactions among ultrasound images, the probe pose and the contact force. The parameters of the model are learned using the data collected from skilled sonographers' demonstrations. Second, a sampling-based strategy is proposed with the learned model to adjust the extracorporeal ultrasound scanning process to guide a newbie sonographer or a robot arm. Finally, the robustness of the proposed framework is validated with the experiments on real data from sonographers.
The Power of Selecting Key Blocks with Local Pre-ranking for Long Document Information Retrieval
Authors: Minghan Li, Diana Nicoleta Popa, Johan Chagnon, Yagmur Gizem Cinar, Eric Gaussier
Date: 2021-11-18 00:00:00
Tasks: Information Retrieval, Retrieval

On a wide range of natural language processing and information retrieval tasks, transformer-based models, particularly pre-trained language models like BERT, have demonstrated tremendous effectiveness. Due to the quadratic complexity of the self-attention mechanism, however, such models have difficulties processing long documents. Recent works dealing with this issue include truncating long documents, in which case one loses potential relevant information, segmenting them into several passages, which may lead to miss some information and high computational complexity when the number of passages is large, or modifying the self-attention mechanism to make it sparser as in sparse-attention models, at the risk again of missing some information. We follow here a slightly different approach in which one first selects key blocks of a long document by local query-block pre-ranking, and then few blocks are aggregated to form a short document that can be processed by a model such as BERT. Experiments conducted on standard Information Retrieval datasets demonstrate the effectiveness of the proposed approach.
Patent Sentiment Analysis to Highlight Patent Paragraphs
Authors: Renukswamy Chikkamath, Vishvapalsinhji Ramsinh Parmar, Christoph Hewel, Markus Endres
Date: 2021-11-06 00:00:00
Tasks: BIG-bench Machine Learning, Sentiment Analysis, text annotation

Given a patent document, identifying distinct semantic annotations is an interesting research aspect. Text annotation helps the patent practitioners such as examiners and patent attorneys to quickly identify the key arguments of any invention, successively providing a timely marking of a patent text. In the process of manual patent analysis, to attain better readability, recognising the semantic information by marking paragraphs is in practice. This semantic annotation process is laborious and time-consuming. To alleviate such a problem, we proposed a novel dataset to train Machine Learning algorithms to automate the highlighting process. The contributions of this work are: i) we developed a multi-class, novel dataset of size 150k samples by traversing USPTO patents over a decade, ii) articulated statistics and distributions of data using imperative exploratory data analysis, iii) baseline Machine Learning models are developed to utilize the dataset to address patent paragraph highlighting task, iv) dataset and codes relating to this task are open-sourced through a dedicated GIT web page: https://github.com/Renuk9390/Patent_Sentiment_Analysis and v) future path to extend this work using Deep Learning and domain specific pre-trained language models to develop a tool to highlight is provided. This work assist patent practitioners in highlighting semantic information automatically and aid to create a sustainable and efficient patent analysis using the aptitude of Machine Learning.
Automated PII Extraction from Social Media for Raising Privacy Awareness: A Deep Transfer Learning Approach
Authors: Yizhi Liu, Fang Yu Lin, MohammadReza Ebrahimi, Weifeng Li, Hsinchun Chen
Date: 2021-11-11 00:00:00
Tasks: Embeddings Evaluation, Feature Engineering, Transfer Learning, Word Embeddings

Internet users have been exposing an increasing amount of Personally Identifiable Information (PII) on social media. Such exposed PII can cause severe losses to the users, and informing users of their PII exposure is crucial to raise their privacy awareness and encourage them to take protective measures. To this end, advanced automatic techniques are needed. While Information Extraction (IE) techniques can be used to extract the PII automatically, Deep Learning (DL)-based IE models alleviate the need for feature engineering and further improve the efficiency. However, DL-based IE models often require large-scale labeled data for training, but PII-labeled social media posts are difficult to obtain due to privacy concerns. Also, these models rely heavily on pre-trained word embeddings, while PII in social media often varies in forms and thus has no fixed representations in pre-trained word embeddings. In this study, we propose the Deep Transfer Learning for PII Extraction (DTL-PIIE) framework to address these two limitations. DTL-PIIE transfers knowledge learned from publicly available PII data to social media to address the problem of rare PII-labeled data. Moreover, our framework leverages Graph Convolutional Networks (GCNs) to incorporate syntactic patterns to guide PIIE without relying on pre-trained word embeddings. Evaluation against benchmark IE models indicates that our approach outperforms state-of-the-art DL-based IE models. Our framework can facilitate various applications, such as PII misuse prediction and privacy risk assessment, protecting the privacy of internet users.
Data-driven synthesis of Robust Invariant Sets and Controllers
Authors: Sampath Kumar Mulagaleti, Alberto Bemporad, Mario Zanon
Date: 2021-11-18 00:00:00
Tasks: Model Predictive Control

This paper presents a method to identify an uncertain linear time-invariant (LTI) prediction model for tube-based Robust Model Predictive Control (RMPC). The uncertain model is determined from a given state-input dataset by formulating and solving a Semidefinite Programming problem (SDP), that also determines a static linear feedback gain and corresponding invariant sets satisfying the inclusions required to guarantee recursive feasibility and stability of the RMPC scheme, while minimizing an identification criterion. As demonstrated through an example, the proposed concurrent approach provides less conservative invariant sets than a sequential approach.
The Strong Integral Input-to-State Stability Property in Dynamical Flow Networks
Authors: Gustav Nilsson, Samuel Coogan
Date: 2021-11-18 00:00:00

Dynamical flow networks serve as macroscopic models for, e.g., transportation networks, queuing networks, and distribution networks. While the flow dynamics in such networks follow the conservation of mass on the links, the outflow from each link is often non-linear due to, e.g., flow capacity constraints and simultaneous service rate constraints. Such non-linear constraints imply a limit on the magnitude of exogenous inflow that is able to be accommodated by the network before it becomes overloaded and its state trajectory diverges. This paper shows how the Strong integral Input-to-State Stability (Strong iISS) property allows for quantifying the effects of the exogenous inflow on the flow dynamics. The Strong iISS property enables a unified stability analysis of classes of dynamical flow networks that were only partly analyzable before, such as networks with cycles, multi-commodity flow networks and networks with non-monotone flow dynamics. We present sufficient conditions on the maximum magnitude of exogenous inflow to guarantee input-to-state stability for a dynamical flow network, and we also present cases when this sufficient condition is necessary. The conditions are exemplified on a few existing dynamical flow network models, specifically, fluid queuing models with time-varying exogenous inflows and multi-commodity flow models.
Evaluating Cybersecurity Risks of Cooperative Ramp Merging in Mixed Traffic Environments
Authors: Xuanpeng Zhao, Ahmed Abdo, Xishun Liao, Matthew J. Barth, Guoyuan Wu
Date: 2021-11-18 00:00:00
Tasks: Management

Connected and Automated Vehicle (CAV) technology has the potential to greatly improve transportation mobility, safety, and energy efficiency. However, ubiquitous vehicular connectivity also opens up the door for cyber-attacks. In this study, we investigate cybersecurity risks of a representative cooperative traffic management application, i.e., highway on-ramp merging, in a mixed traffic environment. We develop threat models with two trajectory spoofing strategies on CAVs to create traffic congestion, and we also devise an attack-resilient strategy for system defense. Furthermore, we leverage VENTOS, a Veins extension simulator made for CAV applications, to evaluate cybersecurity risks of the attacks and performance of the proposed defense strategy. A comprehensive case study is conducted across different traffic congestion levels, penetration rates of CAVs, and attack ratios. As expected, the results show that the performance of mobility decreases up to 55.19% at the worst case when the attack ratio increases, as does safety and energy. With our proposed mitigation defense algorithm, the system's cyber-attack resiliency is greatly improved.
Transmission power policies for energy-efficient wireless control of nonlinear system
Authors: Vineeth S. Varma, Romain Postoyan, Daniel E. Quevedo, Irinel-Constantin Morarescu
Date: 2021-11-18 00:00:00

We present a controller and transmission policy design procedure for nonlinear wireless networked control systems. Our objective is to ensure the stability of the closed-loop system, in a stochastic sense, together with given control performance, while minimizing the average power used for communications. The controller is designed by emulation, i.e., ignoring the network, and the transmission power is given by threshold policies. These policies involve waiting a given amount of time since the last successful transmission instant, as well as requiring that the measured wireless channel gain is above a given threshold, before attempting a new transmission. Two power control laws are investigated: i) a constant power and ii) a power level inversely proportional to the channel gain. We explain how to select the waiting time, the channel threshold and the power level to minimize the induced average communication power, while ensuring the desired control objectives.
A Novel Grid-forming Voltage Control Strategy for Supplying Unbalanced Microgrid Loads Using Inverter-based Resources
Authors: Bei Xu, Victor Paduani, Hui Yu, David Lubkeman, Ning Lu
Date: 2021-11-18 00:00:00
Tasks: Scheduling

This paper presents a novel grid-forming voltage control strategy for a battery energy storage system to maintain balanced three-phase output voltages when serving unbalanced loads. A stationary reference frame ($\alpha \beta$) based control scheme is proposed to regulate positive-sequence and negative-sequence voltages. Compared with the conventional rotating reference frame ($dq$) based control scheme, the proposed scheme shows better dynamic performance. Then, we analyze the system zero-sequence network and propose adding a grounding transformer to the Y/Yg connected output transformer to further reduce the voltage unbalance at the point of common coupling. The simulation results demonstrate the effective performance of the proposed voltage control scheme. Based on the results, the power-voltage unbalance curve is derived for different output transformer configurations to establish the relationship between the power unbalance limit and the voltage unbalance limit for microgrid power scheduling.
Feedback Strategies for Hypersonic Pursuit of a Ground Evader
Authors: Yoonjae Lee, Efstathios Bakolas, Maruthi R. Akella
Date: 2021-11-18 00:00:00

In this paper, we present a game-theoretic feedback terminal guidance law for an autonomous, unpowered hypersonic pursuit vehicle that seeks to intercept an evading ground target whose motion is constrained in a one-dimensional space. We formulate this problem as a pursuit-evasion game whose saddle point solution is in general difficult to compute onboard the hypersonic vehicle due to its highly nonlinear dynamics. To overcome this computational complexity, we linearize the nonlinear hypersonic dynamics around a reference trajectory and subsequently utilize feedback control design techniques from Linear Quadratic Differential Games (LQDGs). In our proposed guidance algorithm, the hypersonic vehicle computes its open-loop optimal state and input trajectories off-line and prior to the commencement of the game. These trajectories are then used to linearize the nonlinear equations of hypersonic motion. Subsequently, using this linearized system model, we formulate an auxiliary two-player zero-sum LQDG which is effective in the neighborhood of the given reference trajectory and derive its feedback saddle point strategy that allows the hypersonic vehicle to modify its trajectory online in response to the target's evasive maneuvers. We provide numerical simulations to showcase the performance of our proposed guidance law.
Combining Trajectory Data with Analytical Lyapunov Functions for Improved Region of Attraction Estimation
Authors: Lucas Lugnani, Morgan Jones, Luís F. C. Alberto, Mathew Peet, Daniel Dotta
Date: 2021-11-17 00:00:00

The increasing uptake of inverter based resources (IBRs) has resulted in many new challenges for power system operators around the world. The high level of complexity of IBR generators makes accurate classical model-based stability analysis a difficult task. This paper proposes a novel methodology for solving the problem of estimating the Region of Attraction (ROA) of a nonlinear system by combining classical model based methods with modern data driven methods. Our method yields certifiable inner approximations of the ROA, typical to that of model based methods, but also harnesses trajectory data to yield an improved accurate ROA estimation. The method is carried out by using analytical Lyapunov functions, such as energy functions, in combination with data that is used to fit a converse Lyapunov function. Our methodology is independent of the function fitting method used. In this work, for implementation purposes, we use Bernstein polynomials to function fit. Several numerical examples of ROA estimation are provided, including the Single Machine Infinite Bus (SMIB) system, a three machine system and the Van-der-Pol system.
Developmental Status and Perspectives for Tissue Engineering in Urology
Authors: Elcin Huseyn
Date: 2021-11-14 00:00:00

Tissue engineering technology and tissue cell-based stem cell research have made great strides in treating tissue and organ damage, correcting tissue and organ dysfunction, and reducing surgical complications. In the past, traditional methods have used biological substitutes for tissue repair materials, while tissue engineering technology has focused on merging sperm cells with biological materials to form biological tissues with the same structure and function as their own tissues. The advantage is that tissue engineering technology can overcome donors. Material procurement restrictions can effectively reduce complications. The aim of studying tissue engineering technology is to find sperm cells and suitable biological materials to replace the original biological functions of tissues and to establish a suitable in vivo microenvironment. This article mainly describes the current developments of tissue engineering in various fields of urology and discusses the future trends of tissue engineering technology in the treatment of complex diseases of the urinary system. The results of the research in this article indicate that while the current clinical studies are relatively few, the good results from existing animal model studies indicate good prospects of tissue engineering technology for the treatment of various urinary tract diseases in the future.
Non-parametric model-based estimation of the effective reproduction number for SARS-CoV-2
Authors: Jacques Hermes, Marcus Rosenblatt, Christian Tönsing, Jens Timmer
Date: 2021-11-18 00:00:00
Tasks: parameter estimation

Viral outbreaks, such as the current COVID-19 pandemic, are commonly described by compartmental models by means of ordinary differential equation (ODE) systems. The parameter values of these ODE models are typically unknown and need to be estimated based on accessible data. In order to describe realistic pandemic scenarios with strongly varying situations, these model parameters need to be assumed as time-dependent. While parameter estimation for the typical case of time-constant parameters does not pose larger issues, the determination of time-dependent parameters, e.g.~the transition rates of compartmental models, remains notoriously difficult, in particular since the function class of these time-dependent parameters is unknown. In this work, we present a novel method which utilizes the Augmented Kalman Smoother in combination with an Expectation-Maximization algorithm to simultaneously estimate all time-dependent parameters in an SIRD compartmental model. This approach only requires incidence data, but no prior knowledge on model parameters or any further assumptions on the function class of the time-dependencies. In contrast to other approaches for the estimation of the time-dependent reproduction number, no assumptions on the parameterization of the serial interval distribution are required. With this method, we are able to adequately describe COVID-19 data in Germany and to give non-parametric model-based time course estimates for the effective reproduction number.