pwc-archive/papers-with-abstracts huggingface dataset:
BiBERT: Accurate Fully Binarized BERT
Authors: Haotong Qin, Yifu Ding, Mingyuan Zhang, Qinghua Yan, Aishan Liu, Qingqing Dang, Ziwei Liu, Xianglong Liu
Date: 2022-03-12 00:00:00
Proceeding: ICLR 2022 4
Tasks: Binarization

The large pre-trained BERT has achieved remarkable performance on Natural Language Processing (NLP) tasks but is also computation and memory expensive. As one of the powerful compression approaches, binarization extremely reduces the computation and memory consumption by utilizing 1-bit parameters and bitwise operations. Unfortunately, the full binarization of BERT (i.e., 1-bit weight, embedding, and activation) usually suffer a significant performance drop, and there is rare study addressing this problem. In this paper, with the theoretical justification and empirical analysis, we identify that the severe performance drop can be mainly attributed to the information degradation and optimization direction mismatch respectively in the forward and backward propagation, and propose BiBERT, an accurate fully binarized BERT, to eliminate the performance bottlenecks. Specifically, BiBERT introduces an efficient Bi-Attention structure for maximizing representation information statistically and a Direction-Matching Distillation (DMD) scheme to optimize the full binarized BERT accurately. Extensive experiments show that BiBERT outperforms both the straightforward baseline and existing state-of-the-art quantized BERTs with ultra-low bit activations by convincing margins on the NLP benchmark. As the first fully binarized BERT, our method yields impressive 56.3 times and 31.2 times saving on FLOPs and model size, demonstrating the vast advantages and potential of the fully binarized BERT model in real-world resource-constrained scenarios.
MarkBERT: Marking Word Boundaries Improves Chinese BERT
Authors: Linyang Li, Yong Dai, Duyu Tang, Xipeng Qiu, Zenglin Xu, Shuming Shi
Date: 2022-03-12 00:00:00
Tasks: Chinese Named Entity Recognition, named-entity-recognition, Named Entity Recognition (NER), POS, Semantic Similarity, Semantic Textual Similarity, Sentence, TAG, text-classification, Text Classification

We present a Chinese BERT model dubbed MarkBERT that uses word information in this work. Existing word-based BERT models regard words as basic units, however, due to the vocabulary limit of BERT, they only cover high-frequency words and fall back to character level when encountering out-of-vocabulary (OOV) words. Different from existing works, MarkBERT keeps the vocabulary being Chinese characters and inserts boundary markers between contiguous words. Such design enables the model to handle any words in the same way, no matter they are OOV words or not. Besides, our model has two additional benefits: first, it is convenient to add word-level learning objectives over markers, which is complementary to traditional character and sentence-level pretraining tasks; second, it can easily incorporate richer semantics such as POS tags of words by replacing generic markers with POS tag-specific markers. With the simple markers insertion, MarkBERT can improve the performances of various downstream tasks including language understanding and sequence labeling. \footnote{All the codes and models will be made publicly available at \url{https://github.com/daiyongya/markbert}}
Cross-lingual Inference with A Chinese Entailment Graph
Authors: Tianyi Li, Sabine Weber, Mohammad Javad Hosseini, Liane Guillou, Mark Steedman
Date: 2022-03-11 00:00:00
Proceeding: Findings (ACL) 2022 5
Tasks: Entity Typing, Question Answering, Relation, Relation Extraction

Predicate entailment detection is a crucial task for question-answering from text, where previous work has explored unsupervised learning of entailment graphs from typed open relation triples. In this paper, we present the first pipeline for building Chinese entailment graphs, which involves a novel high-recall open relation extraction (ORE) method and the first Chinese fine-grained entity typing dataset under the FIGER type ontology. Through experiments on the Levy-Holt dataset, we verify the strength of our Chinese entailment graph, and reveal the cross-lingual complementarity: on the parallel Levy-Holt dataset, an ensemble of Chinese and English entailment graphs outperforms both monolingual graphs, and raises unsupervised SOTA by 4.7 AUC points.
BioSimulators: a central registry of simulation engines and services for recommending specific tools
Authors: Bilal Shaikh, Lucian P. Smith, Dan Vasilescu, Gnaneswara Marupilla, Michael Wilson, Eran Agmon, Henry Agnew, Steven S. Andrews, Azraf Anwar, Moritz E. Beber, Frank T. Bergmann, David Brooks, Lutz Brusch, Laurence Calzone, Kiri Choi, Joshua Cooper, John Detloff, Brian Drawert, Michel Dumontier, G. Bard Ermentrout, James R. Faeder, Andrew P. Freiburger, Fabian Fröhlich, Akira Funahashi, Alan Garny, John H. Gennari, Padraig Gleeson, Anne Goelzer, Zachary Haiman, Joseph L. Hellerstein, Stefan Hoops, Jon C. Ison, Diego Jahn, Henry V. Jakubowski, Ryann Jordan, Matúš Kalaš, Matthias König, Wolfram Liebermeister, Synchon Mandal, Robert McDougal, J. Kyle Medley, Pedro Mendes, Robert Müller, Chris J. Myers, Aurelien Naldi, Tung V. N. Nguyen, David P. Nickerson, Brett G. Olivier, Drashti Patoliya, Loïc Paulevé, Linda R. Petzold, Ankita Priya, Anand K. Rampadarath, Johann M. Rohwer, Ali S. Saglam, Dilawar Singh, Ankur Sinha, Jacky Snoep, Hugh Sorby, Ryan Spangler, Jörn Starruß, Payton J. Thomas, David van Niekerk, Daniel Weindl, Fengkai Zhang, Anna Zhukova, Arthur P. Goldberg, Michael L. Blinov, Herbert M. Sauro, Ion I. Moraru, Jonathan R. Karr
Date: 2022-03-13 00:00:00

Computational models have great potential to accelerate bioscience, bioengineering, and medicine. However, it remains challenging to reproduce and reuse simulations, in part, because the numerous formats and methods for simulating various subsystems and scales remain siloed by different software tools. For example, each tool must be executed through a distinct interface. To help investigators find and use simulation tools, we developed BioSimulators (https://biosimulators.org), a central registry of the capabilities of simulation tools and consistent Python, command-line, and containerized interfaces to each version of each tool. The foundation of BioSimulators is standards, such as CellML, SBML, SED-ML, and the COMBINE archive format, and validation tools for simulation projects and simulation tools that ensure these standards are used consistently. To help modelers find tools for particular projects, we have also used the registry to develop recommendation services. We anticipate that BioSimulators will help modelers exchange, reproduce, and combine simulations.
Model Positionality and Computational Reflexivity: Promoting Reflexivity in Data Science
Authors: Scott Allen Cambo, Darren Gergle
Date: 2022-03-08 00:00:00
Tasks: Position

Data science and machine learning provide indispensable techniques for understanding phenomena at scale, but the discretionary choices made when doing this work are often not recognized. Drawing from qualitative research practices, we describe how the concepts of positionality and reflexivity can be adapted to provide a framework for understanding, discussing, and disclosing the discretionary choices and subjectivity inherent to data science work. We first introduce the concepts of model positionality and computational reflexivity that can help data scientists to reflect on and communicate the social and cultural context of a model's development and use, the data annotators and their annotations, and the data scientists themselves. We then describe the unique challenges of adapting these concepts for data science work and offer annotator fingerprinting and position mining as promising solutions. Finally, we demonstrate these techniques in a case study of the development of classifiers for toxic commenting in online communities.
AI agents for facilitating social interactions and wellbeing
Authors: Hiro Taiyo Hamada, Ryota Kanai
Date: 2022-02-26 00:00:00

Wellbeing AI has been becoming a new trend in individuals' mental health, organizational health, and flourishing our societies. Various applications of wellbeing AI have been introduced to our daily lives. While social relationships within groups are a critical factor for wellbeing, the development of wellbeing AI for social interactions remains relatively scarce. In this paper, we provide an overview of the mediative role of AI-augmented agents for social interactions. First, we discuss the two-dimensional framework for classifying wellbeing AI: individual/group and analysis/intervention. Furthermore, wellbeing AI touches on intervening social relationships between human-human interactions since positive social relationships are key to human wellbeing. This intervention may raise technical and ethical challenges. We discuss opportunities and challenges of the relational approach with wellbeing AI to promote wellbeing in our societies.
Identifying the root cause of cable network problems with machine learning
Authors: Georg Heiler, Thassilo Gadermaier, Thomas Haider, Allan Hanbury, Peter Filzmoser
Date: 2022-03-09 00:00:00
Tasks: BIG-bench Machine Learning

Good quality network connectivity is ever more important. For hybrid fiber coaxial (HFC) networks, searching for upstream high noise in the past was cumbersome and time-consuming. Even with machine learning due to the heterogeneity of the network and its topological structure, the task remains challenging. We present the automation of a simple business rule (largest change of a specific value) and compare its performance with state-of-the-art machine-learning methods and conclude that the precision@1 can be improved by 2.3 times. As it is best when a fault does not occur in the first place, we secondly evaluate multiple approaches to forecast network faults, which would allow performing predictive maintenance on the network.
Cluster Assignment in Multi-Agent Systems
Authors: Miel Sharf, Daniel Zelazo
Date: 2022-03-13 00:00:00

We study cluster assignment in multi-agent networks. We consider homogeneous diffusive networks, and focus on design of the graph that ensures the system will converge to a prescribed cluster configuration, i.e., specifying the number of clusters and agents within each cluster. Leveraging recent results from cluster synthesis, we show that it is possible to design an oriented graph such that the action of the automorphism group of the graph has orbits of predetermined sizes, guaranteeing that the network will converge to the prescribed cluster configuration. We provide upper and lower bounds on the number of edges that are needed to construct these graphs along with a constructive approach for generating these graphs. We support our analysis with some numerical examples.
Beckmann's approach to multi-item multi-bidder auctions
Authors: Alexander V. Kolesnikov, Fedor Sandomirskiy, Aleh Tsyvinski, Alexander P. Zimin
Date: 2022-03-14 00:00:00

We consider the problem of revenue-maximizing Bayesian auction design with several bidders having independent private values over several items. We show that it can be reduced to the problem of continuous optimal transportation introduced by Beckmann (1952) where the optimal transportation flow generalizes the concept of ironed virtual valuations to the multi-item setting. We establish the strong duality between the two problems and the existence of solutions. The results rely on insights from majorization and optimal transportation theories and on the characterization of feasible interim mechanisms by Hart and Reny (2015).
Efficiency in Random Resource Allocation and Social Choice
Authors: Federico Echenique, Joseph Root, Fedor Sandomirskiy
Date: 2022-03-12 00:00:00

We study efficiency in general collective choice problems where agents have ordinal preferences and randomization is allowed. We explore the structure of preference profiles where ex-ante and ex-post efficiency coincide, offer a unifying perspective on the known results, and give several new characterizations. The results have implications for well-studied mechanisms including random serial dictatorship and a number of specific environments, including the dichotomous, single-peaked, and social choice domains.
Optimal Aggregation Strategies for Social Learning over Graphs
Authors: Ping Hu, Virginia Bordignon, Stefan Vlaski, Ali H. Sayed
Date: 2022-03-14 00:00:00
Tasks: Decision Making

Adaptive social learning is a useful tool for studying distributed decision-making problems over graphs. This paper investigates the effect of combination policies on the performance of adaptive social learning strategies. Using large-deviation analysis, it first derives a bound on the steady-state error probability and characterizes the optimal selection for the Perron eigenvectors of the combination policies. It subsequently studies the effect of the combination policy on the transient behavior of the learning strategy by estimating the adaptation time in the low signal-to-noise ratio regime. In the process, it is discovered that, interestingly, the influence of the combination policy on the transient behavior is insignificant, and thus it is more critical to employ policies that enhance the steady-state performance. The theoretical conclusions are illustrated by means of computer simulations.
Low-Complexity Multicast Beamforming for Millimeter Wave Communications
Authors: Zhaohui Li, Chenhao Qi, Geoffrey Ye Li
Date: 2022-03-12 00:00:00

To develop a low-complexity multicast beamforming method for millimeter wave communications, we first propose a channel gain estimation method in this article. We use the beam sweeping to find the best codeword and its two neighboring codewords to form a composite beam. We then estimate the channel gain based on the composite beam, which is computed off-line by minimizing the variance of beam gain within beam coverage. With the estimated channel gain, we propose a multicast beamforming design method under the max-min fair (MMF) criterion. To reduce the computational complexity, we divide the large antenna array into several small-size sub-arrays, where the size of each sub-array is determined by the estimated channel gain. In particular, we introduce a phase factor for each sub-array to explore additional degree of freedom for the considered problem. Simulation results show that the proposed multicast beamforming design method can substantially reduce the computational complexity with little performance sacrifice compared to the existing methods.
Hybrid Beamforming for Millimeter Wave MIMO Integrated Sensing and Communications
Authors: Chenhao Qi, Wei Ci, Jinming Zhang, Xiaohu You
Date: 2022-03-12 00:00:00
Tasks: ISAC

In this letter, we consider hybrid beamforming for millimeter wave (mmWave) MIMO integrated sensing and communications (ISAC). We design the transmit beam of a dual-functional radar-communication (DFRC) base station (BS), aiming at approaching the objective radar beam pattern, subject to the constraints of the signal to interference-plus-noise ratio (SINR) of communication users and total transmission power of the DFRC BS. To provide additional degree of freedom for the beam design problem, we introduce a phase vector to the objective beam pattern and propose an alternating minimization method to iteratively optimize the transmit beam and the phase vector, which involves second-order cone programming and constrained least squared estimation, respectively. Then based on the designed transmit beam, we determine the analog beamformer and digital beamformer subject to the constant envelop constraint of phase shifter network in mmWave MIMO, still using the alternating minimization method. Simulation results show that under the same SINR constraint of communication users, larger antenna array can achieve better radar beam quality.
An Observer-Based Composite Identifier for Online Estimation of the Thevenin Equivalent Parameters of a Power System
Authors: Daniele Zonetti, Romeo Ortega, Rafael Cisneros, Alexey Bobtsov, Fernando Mancilla-David, Oriol Gomis-Bellmunt
Date: 2022-03-14 00:00:00

We consider a Th\'evenin equivalent circuit capturing the dynamics of a power grid as seen from the point of common coupling with a power electronic converter, and provide a solution to the problem of online identification of the corresponding circuit parameters. For this purpose, we first derive a linear regression model in the conventional abc coordinates and next design a bounded observer-based composite identifier that requires local measurements and knowledge of the grid frequency only. An extension that guarantees exponential convergence of the estimates, under the additional assumption of knowledge of the grid X/R ratio, is further provided. The performance of the proposed identifier, which subsumes a conventional gradient descent algorithm, is illustrated via detailed computer simulations.
Allocation of spinning reserves for autonomous grids subject to frequency stability constraints and short-term solar power variations
Authors: Erick Fernando Alves, Louis Polleux, Gilles Guerassimoff, Magnus Korpås, Elisabetta Tedeschi
Date: 2022-03-14 00:00:00
Tasks: Management

Low-inertia, isolated power systems face the problem of resiliency to power variations. The integration of renewable energy sources, such as wind and solar photovoltaic, pushes the boundaries of this issue further. Higher shares of renewables requires better evaluations of electrical system stability, to avoid severe safety and economic consequences. Accounting for frequency stability requirements and allocating proper spinning reserves, therefore becomes a topic of pivotal importance in the long-term planning and operational management of power systems. In this paper, dynamic frequency constraints are proposed to ensure resiliency during short-term power variations due to, for example, wind gusts or cloud passage. The use of the proposed constraints is exemplified in a case study, the constraints being integrated into a mixed-integer linear programming algorithm for sizing the optimal capacities of solar photovoltaic and battery energy storage resources in an isolated industrial plant. Outcomes of this case study show that reductions in the levelized cost of energy and carbon emissions can be overestimated by 8.0% and 10.8% respectively, where frequency constraints are neglected. The proposed optimal sizing is validated using time-domain simulations of the case study. The results indicate that this optimal system is frequency stable under the worst-case contingency.
Model predictive control and moving horizon estimation for adaptive optimal bolus feeding in high-throughput cultivation of \textit{E. coli}
Authors: Jong Woo Kim, Niels Krausch, Judit Aizpuru, Tilman Barz, Sergio Lucia, Peter Neubauer, Mariano Nicolas Cruz Bournazou
Date: 2022-03-14 00:00:00
Tasks: Model Predictive Control

We discuss the application of a nonlinear model predictive control (MPC) and a moving horizon estimation (MHE) to achieve an optimal operation of \textit{E. coli} fed-batch cultivations with intermittent bolus feeding. 24 parallel experiments were considered in a high-throughput microbioreactor platform at a 10 mL scale. The robotic island in question can run up to 48 fed-batch processes in parallel with automated liquid handling and online and at-line analytics. The implementation of the model-based monitoring and control framework reveals that there are mainly three challenges that need to be addressed; First, the inputs are given in an instantaneous pulsed form by bolus injections, second, online and at-line measurement frequencies are severely imbalanced, and third, optimization for the distinctive multiple reactors can be either parallelized or integrated. We address these challenges by incorporating the concept of impulsive control systems, formulating multi-rate MHE with identifiability analysis, and suggesting criteria for deciding the reactor configuration. In this study, we present the key elements and background theory of the implementation with \textit{in silico} simulations for bacterial fed-batch cultivation.
Incremental Control System Design and Flight Tests of a Micro-Coaxial Rotor UAV
Authors: Z. C. Liu, Y. F. Zhang, Z. D. zhang, H. X. Chen
Date: 2022-03-14 00:00:00

In this paper, the incremental nonlinear dynamic inversion (INDI) method is applied to the control system design of coaxial rotor UAVs. The aerodynamic uncertainty and anti-disturbance problems are solved in the control system design. The designed controller gives the UAV excellent flight performance and control robustness. An incremental gain method (IGM) is proposed for the delay problem of the state derivative. This method has the advantages of less calculation load and simple parameter adjustment, which provides excellent convenience for applying INDI controllers to coaxial rotor UAVs. The principle of IGM is demonstrated by the stability analysis method of the discrete system, and the parameter selection strategy of the IGM is analyzed in detail by simulation. The advantages of the controller design method are verified by comparative flight tests of nonlinear dynamic inversion (NDI) and INDI. The experimental results show that the average trajectory tracking error of INDI is only 58.3% of that of NDI under the same wind speed. When the angular acceleration delay is less than 0.06 s, IGM can easily keep the system stable. In addition, INDI has better robustness under obvious model errors and strong input disturbance.
A novel constraint tightening approach for robust data-driven predictive control
Authors: Christian Klöppelt, Julian Berberich, Frank Allgöwer, Matthias A. Müller
Date: 2022-03-14 00:00:00
Tasks: LEMMA, Model Predictive Control

In this paper, we present a data-driven model predictive control (MPC) scheme that is capable of stabilizing unknown linear time-invariant systems under the influence of process disturbances. To this end, Willems' lemma is used to predict the future behavior of the system. This allows the entire scheme to be set up using only a priori measured data and knowledge of an upper bound on the system order. First, we develop a state-feedback MPC scheme, based on input-state data, which guarantees closed-loop practical exponential stability and recursive feasibility as well as closed-loop constraint satisfaction. The scheme is extended by a suitable constraint tightening, which can also be constructed using only data. In order to control a priori unstable systems, the presented scheme contains a pre-stabilizing controller and an associated input constraint tightening. We first present the proposed data-driven MPC scheme for the case of full state measurements, and also provide extensions for obtaining similar closed-loop guarantees in case of output feedback. The presented scheme is applied to a numerical example.
A Compositional Algorithm for the Conflict-Free Electric Vehicle Routing Problem
Authors: Sabino Francesco Roselli, Per-Lage Götvall, Martin Fabian, Knut Åkesson
Date: 2022-03-14 00:00:00
Tasks: Combinatorial Optimization

The Conflict-Free Electric Vehicle Routing Problem (CF-EVRP) is an extension of the Vehicle Routing Problem (VRP), a combinatorial optimization problem of designing routes for vehicles to visit customers such that a cost function, typically the number of vehicles or the total travelled distance, is minimized. The problem finds many logistics applications, particularly for highly automated logistic systems for material handling. The CF-EVRP involves constraints such as time windows on the delivery to the customers, limited operating range of the vehicles, and limited capacity on the number of vehicles that a road segment can accommodate at the same time. In this paper, the compositional algorithm ComSat for solving the CF-EVRP is presented. The algorithm iterates through the sub-problems until a globally feasible solution is found. The proposed algorithm is implemented using an optimizing SMT-solver and is evaluated against an implementation of a previously presented monolithic model. The soundness and completeness of the algorithm are proven, and it is benchmarked on a set of generated problems and found to be able to solve problems of industrial size.