OpenRaiser/Intern-Atlas huggingface dataset:
Computational bioacoustics with deep learning: a review and roadmap
Venue: arXiv cs.SD 2021
Year: 2021

Animal vocalisations and natural soundscapes are fascinating objects of study, and contain valuable evidence about animal behaviours, populations and ecosystems. They are studied in bioacoustics and ecoacoustics, with signal processing and analysis an important component. Computational bioacoustics has accelerated in recent decades due to the growth of affordable digital sound recording devices, and to huge progress in informatics such as big data, signal processing and machine learning. Methods are inherited from the wider field of deep learning, including speech and image processing. However, the tasks, demands and data characteristics are often different from those addressed in speech or music analysis. There remain unsolved problems, and tasks for which evidence is surely present in many acoustic signals, but not yet realised. In this paper I perform a review of the state of the art in deep learning for computational bioacoustics, aiming to clarify key concepts and identify and analyse knowledge gaps. Based on this, I offer a subjective but principled roadmap for computational bioacoustics with deep learning: topics that the community should aim to address, in order to make the most of future developments in AI and informatics, and to use audio data in answering zoological and ecological questions.
VirtualCube: An Immersive 3D Video Communication System
Venue: arXiv cs.CV 2021
Year: 2021

The VirtualCube system is a 3D video conference system that attempts to overcome some limitations of conventional technologies. The key ingredient is VirtualCube, an abstract representation of a real-world cubicle instrumented with RGBD cameras for capturing the 3D geometry and texture of a user. We design VirtualCube so that the task of data capturing is standardized and significantly simplified, and everything can be built using off-the-shelf hardware. We use VirtualCubes as the basic building blocks of a virtual conferencing environment, and we provide each VirtualCube user with a surrounding display showing life-size videos of remote participants. To achieve real-time rendering of remote participants, we develop the V-Cube View algorithm, which uses multi-view stereo for more accurate depth estimation and Lumi-Net rendering for better rendering quality. The VirtualCube system correctly preserves the mutual eye gaze between participants, allowing them to establish eye contact and be aware of who is visually paying attention to them. The system also allows a participant to have side discussions with remote participants as if they were in the same room. Finally, the system sheds lights on how to support the shared space of work items (e.g., documents and applications) and track the visual attention of participants to work items.
Attentive Contextual Carryover for Multi-Turn End-to-End Spoken Language Understanding
Venue: arXiv cs.CL 2021
Year: 2021

Recent years have seen significant advances in end-to-end (E2E) spoken language understanding (SLU) systems, which directly predict intents and slots from spoken audio. While dialogue history has been exploited to improve conventional text-based natural language understanding systems, current E2E SLU approaches have not yet incorporated such critical contextual signals in multi-turn and task-oriented dialogues. In this work, we propose a contextual E2E SLU model architecture that uses a multi-head attention mechanism over encoded previous utterances and dialogue acts (actions taken by the voice assistant) of a multi-turn dialogue. We detail alternative methods to integrate these contexts into the state-ofthe-art recurrent and transformer-based models. When applied to a large de-identified dataset of utterances collected by a voice assistant, our method reduces average word and semantic error rates by 10.8% and 12.6%, respectively. We also present results on a publicly available dataset and show that our method significantly improves performance over a noncontextual baseline
Adaptation through prediction: multisensory active inference torque control
Venue: arXiv cs.RO 2021
Year: 2021

Adaptation to external and internal changes is major for robotic systems in uncertain environments. Here we present a novel multisensory active inference torque controller for industrial arms that shows how prediction can be used to resolve adaptation. Our controller, inspired by the predictive brain hypothesis, improves the capabilities of current active inference approaches by incorporating learning and multimodal integration of low and high-dimensional sensor inputs (e.g., raw images) while simplifying the architecture. We performed a systematic evaluation of our model on a 7DoF Franka Emika Panda robot arm by comparing its behavior with previous active inference baselines and classic controllers, analyzing both qualitatively and quantitatively adaptation capabilities and control accuracy. Results showed improved control accuracy in goal-directed reaching with high noise rejection due to multimodal filtering, and adaptability to dynamical inertial changes, elasticity constraints and human disturbances without the need to relearn the model nor parameter retuning.
Rational Pavelka logic: the best among three worlds?
Venue: arXiv cs.LO 2021
Year: 2021

This comparative survey explores three formal approaches to reasoning with partly true statements and degrees of truth, within the family of Łukasiewicz logic. These approaches are represented by infinite-valued Łukasiewicz logic (Ł), Rational Pavelka logic (RPL) and a logic with graded formulas that we refer to as Graded Rational Pavelka logic (GRPL). Truth constants for all rationals between $0$ and $1$ are used as a technical means to calibrate degrees of truth. Łukasiewicz logic ostensibly features no truth constants except $0$ and $1$; Rational Pavelka logic includes constants in the basic language, with suitable axioms; Graded Rational Pavelka logic works with graded formulas and proofs, following the original intent of Pavelka, inspired by Goguen's work. Historically, Pavelka's papers precede the definition of GRPL, which in turn precedes RPL; retrieving these steps, we discuss how these formal systems naturally evolve from each other, and we also recall how this process has been a somewhat contentious issue in the realm of Łukasiewicz logic. This work can also be read as a case study in logics, their fragments, and the relationship of the fragments to a logic.
Mean-square-error-based secondary source placement in sound field synthesis with prior information on desired field
Venue: arXiv cs.SD 2021
Year: 2021

A method of optimizing secondary source placement in sound field synthesis is proposed. Such an optimization method will be useful when the allowable placement region and available number of loudspeakers are limited. We formulate a mean-square-error-based cost function, incorporating the statistical properties of possible desired sound fields, for general linear-least-squares-based sound field synthesis methods, including pressure matching and (weighted) mode matching, whereas most of the current methods are applicable only to the pressure-matching method. An efficient greedy algorithm for minimizing the proposed cost function is also derived. Numerical experiments indicated that a high reproduction accuracy can be achieved by the placement optimized by the proposed method compared with the empirically used regular placement.
BScNets: Block Simplicial Complex Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Simplicial neural networks (SNN) have recently emerged as the newest direction in graph learning which expands the idea of convolutional architectures from node space to simplicial complexes on graphs. Instead of pre-dominantly assessing pairwise relations among nodes as in the current practice, simplicial complexes allow us to describe higher-order interactions and multi-node graph structures. By building upon connection between the convolution operation and the new block Hodge-Laplacian, we propose the first SNN for link prediction. Our new Block Simplicial Complex Neural Networks (BScNets) model generalizes the existing graph convolutional network (GCN) frameworks by systematically incorporating salient interactions among multiple higher-order graph structures of different dimensions. We discuss theoretical foundations behind BScNets and illustrate its utility for link prediction on eight real-world and synthetic datasets. Our experiments indicate that BScNets outperforms the state-of-the-art models by a significant margin while maintaining low computation costs. Finally, we show utility of BScNets as the new promising alternative for tracking spread of infectious diseases such as COVID-19 and measuring the effectiveness of the healthcare risk mitigation strategies.
Why Are You Weird? Infusing Interpretability in Isolation Forest for Anomaly Detection
Venue: arXiv cs.LG 2021
Year: 2021

Anomaly detection is concerned with identifying examples in a dataset that do not conform to the expected behaviour. While a vast amount of anomaly detection algorithms exist, little attention has been paid to explaining why these algorithms flag certain examples as anomalies. However, such an explanation could be extremely useful to anyone interpreting the algorithms' output. This paper develops a method to explain the anomaly predictions of the state-of-the-art Isolation Forest anomaly detection algorithm. The method outputs an explanation vector that captures how important each attribute of an example is to identifying it as anomalous. A thorough experimental evaluation on both synthetic and real-world datasets shows that our method is more accurate and more efficient than most contemporary state-of-the-art explainability methods.
Efficient Differentially Private Secure Aggregation for Federated Learning via Hardness of Learning with Errors
Venue: arXiv cs.CR 2021
Year: 2021

Federated machine learning leverages edge computing to develop models from network user data, but privacy in federated learning remains a major challenge. Techniques using differential privacy have been proposed to address this, but bring their own challenges -- many require a trusted third party or else add too much noise to produce useful models. Recent advances in \emph{secure aggregation} using multiparty computation eliminate the need for a third party, but are computationally expensive especially at scale. We present a new federated learning protocol that leverages a novel differentially private, malicious secure aggregation protocol based on techniques from Learning With Errors. Our protocol outperforms current state-of-the art techniques, and empirical results show that it scales to a large number of parties, with optimal accuracy for any differentially private federated learning scheme.
Locally Fair Partitioning
Venue: arXiv cs.DS 2021
Year: 2021

We model the societal task of redistricting political districts as a partitioning problem: Given a set of $n$ points in the plane, each belonging to one of two parties, and a parameter $k$, our goal is to compute a partition $Π$ of the plane into regions so that each region contains roughly $σ= n/k$ points. $Π$ should satisfy a notion of ''local'' fairness, which is related to the notion of core, a well-studied concept in cooperative game theory. A region is associated with the majority party in that region, and a point is unhappy in $Π$ if it belongs to the minority party. A group $D$ of roughly $σ$ contiguous points is called a deviating group with respect to $Π$ if majority of points in $D$ are unhappy in $Π$. The partition $Π$ is locally fair if there is no deviating group with respect to $Π$. This paper focuses on a restricted case when points lie in $1$D. The problem is non-trivial even in this case. We consider both adversarial and ''beyond worst-case" settings for this problem. For the former, we characterize the input parameters for which a locally fair partition always exists; we also show that a locally fair partition may not exist for certain parameters. We then consider input models where there are ''runs'' of red and blue points. For such clustered inputs, we show that a locally fair partition may not exist for certain values of $σ$, but an approximate locally fair partition exists if we allow some regions to have smaller sizes. We finally present a polynomial-time algorithm for computing a locally fair partition if one exists.
HVH: Learning a Hybrid Neural Volumetric Representation for Dynamic Hair Performance Capture
Venue: arXiv cs.CV 2021
Year: 2021

Capturing and rendering life-like hair is particularly challenging due to its fine geometric structure, the complex physical interaction and its non-trivial visual appearance.Yet, hair is a critical component for believable avatars. In this paper, we address the aforementioned problems: 1) we use a novel, volumetric hair representation that is com-posed of thousands of primitives. Each primitive can be rendered efficiently, yet realistically, by building on the latest advances in neural rendering. 2) To have a reliable control signal, we present a novel way of tracking hair on the strand level. To keep the computational effort manageable, we use guide hairs and classic techniques to expand those into a dense hood of hair. 3) To better enforce temporal consistency and generalization ability of our model, we further optimize the 3D scene flow of our representation with multi-view optical flow, using volumetric ray marching. Our method can not only create realistic renders of recorded multi-view sequences, but also create renderings for new hair configurations by providing new control signals. We compare our method with existing work on viewpoint synthesis and drivable animation and achieve state-of-the-art results. Please check out our project website at https://ziyanw1.github.io/hvh/.
Branching Strategy Selection Approach Based on Vivification Ratio
Venue: arXiv cs.AI 2021
Year: 2021

The two most effective branching strategies LRB and VSIDS perform differently on different types of instances. Generally, LRB is more effective on crafted instances, while VSIDS is more effective on application ones. However, distinguishing the types of instances is difficult. To overcome this drawback, we propose a branching strategy selection approach based on the vivification ratio. This approach uses the LRB branching strategy more to solve the instances with a very low vivification ratio. We tested the instances from the main track of SAT competitions in recent years. The results show that the proposed approach is robust and it significantly increases the number of solved instances. It is worth mentioning that, with the help of our approach, the solver Maple\_CM can solve more than 16 instances for the benchmark from the 2020 SAT competition.
Generating Fluent Fact Checking Explanations with Unsupervised Post-Editing
Venue: arXiv cs.CL 2021
Year: 2021

Fact-checking systems have become important tools to verify fake and misguiding news. These systems become more trustworthy when human-readable explanations accompany the veracity labels. However, manual collection of such explanations is expensive and time-consuming. Recent works frame explanation generation as extractive summarization, and propose to automatically select a sufficient subset of the most important facts from the ruling comments (RCs) of a professional journalist to obtain fact-checking explanations. However, these explanations lack fluency and sentence coherence. In this work, we present an iterative edit-based algorithm that uses only phrase-level edits to perform unsupervised post-editing of disconnected RCs. To regulate our editing algorithm, we use a scoring function with components including fluency and semantic preservation. In addition, we show the applicability of our approach in a completely unsupervised setting. We experiment with two benchmark datasets, LIAR-PLUS and PubHealth. We show that our model generates explanations that are fluent, readable, non-redundant, and cover important information for the fact check.
Isometric Hamming embeddings of weighted graphs
Venue: arXiv cs.DM 2021
Year: 2021

A mapping $α: V(G) \to V(H)$ from the vertex set of one graph $G$ to another graph $H$ is an isometric embedding if the shortest path distance between any two vertices in $G$ equals the distance between their images in $H$. Here, we consider isometric embeddings of a weighted graph $G$ into unweighted Hamming graphs, called Hamming embeddings, when $G$ satisfies the property that every edge is a shortest path between its endpoints. Using a Cartesian product decomposition of $G$ called its pseudofactorization, we show that every Hamming embedding of $G$ may be partitioned into Hamming embeddings for each irreducible pseudofactor graph of $G$, which we call its canonical partition. This implies that $G$ permits a Hamming embedding if and only if each of its irreducible pseudofactors is Hamming embeddable. This result extends prior work on unweighted graphs that showed that an unweighted graph permits a Hamming embedding if and only if each irreducible pseudofactor is a complete graph. When a graph $G$ has nontrivial pseudofactors, determining whether $G$ has a Hamming embedding can be simplified to checking embeddability of two or more smaller graphs.
Designing weighted and multiplex networks for deep learning user geolocation in Twitter
Venue: arXiv cs.SI 2021
Year: 2021

Predicting the geographical location of users of social media like Twitter has found several applications in health surveillance, emergency monitoring, content personalization, and social studies in general. In this work we contribute to the research in this area by designing and evaluating new methods based on the literature of weighted multigraphs combined with state-of-the-art deep learning techniques. The explored methods depart from a similar underlying structure (that of an extended mention and/or follower network) but use different information processing strategies, e.g., information diffusion through transductive and inductive algorithms -- RGCNs and GraphSAGE, respectively -- and node embeddings with Node2vec+. These graphs are then combined with attention mechanisms to incorporate the users' text view into the models. We assess the performance of each of these methods and compare them to baseline models in the publicly available Twitter-US dataset; we also make a new dataset available based on a large Twitter capture in Latin America. Finally, our work discusses the limitations and validity of the comparisons among methods in the context of different label definitions and metrics.
Survey of Generative Methods for Social Media Analysis
Venue: arXiv cs.SI 2021
Year: 2021

This survey draws a broad-stroke, panoramic picture of the State of the Art (SoTA) of the research in generative methods for the analysis of social media data. It fills a void, as the existing survey articles are either much narrower in their scope or are dated. We included two important aspects that currently gain importance in mining and modeling social media: dynamics and networks. Social dynamics are important for understanding the spreading of influence or diseases, formation of friendships, the productivity of teams, etc. Networks, on the other hand, may capture various complex relationships providing additional insight and identifying important patterns that would otherwise go unnoticed.
NEORL: NeuroEvolution Optimization with Reinforcement Learning
Venue: arXiv cs.NE 2021
Year: 2021

We present an open-source Python framework for NeuroEvolution Optimization with Reinforcement Learning (NEORL) developed at the Massachusetts Institute of Technology. NEORL offers a global optimization interface of state-of-the-art algorithms in the field of evolutionary computation, neural networks through reinforcement learning, and hybrid neuroevolution algorithms. NEORL features diverse set of algorithms, user-friendly interface, parallel computing support, automatic hyperparameter tuning, detailed documentation, and demonstration of applications in mathematical and real-world engineering optimization. NEORL encompasses various optimization problems from combinatorial, continuous, mixed discrete/continuous, to high-dimensional, expensive, and constrained engineering optimization. NEORL is tested in variety of engineering applications relevant to low carbon energy research in addressing solutions to climate change. The examples include nuclear reactor control and fuel cell power production. The results demonstrate NEORL competitiveness against other algorithms and optimization frameworks in the literature, and a potential tool to solve large-scale optimization problems. More examples and benchmarking of NEORL can be found here: https://neorl.readthedocs.io/en/latest/index.html
Differentially Private Data Publication with Multi-level Data Utility
Venue: arXiv cs.CR 2021
Year: 2021

Conventional private data publication mechanisms aim to retain as much data utility as possible while ensuring sufficient privacy protection on sensitive data. Such data publication schemes implicitly assume that all data analysts and users have the same data access privilege levels. However, it is not applicable for the scenario that data users often have different levels of access to the same data, or different requirements of data utility. The multi-level privacy requirements for different authorization levels pose new challenges for private data publication. Traditional PPDP mechanisms only publish one perturbed and private data copy satisfying some privacy guarantee to provide relatively accurate analysis results. To find a good tradeoff between privacy preservation level and data utility itself is a hard problem, let alone achieving multi-level data utility on this basis. In this paper, we address this challenge in proposing a novel framework of data publication with compressive sensing supporting multi-level utility-privacy tradeoffs, which provides differential privacy. Specifically, we resort to compressive sensing (CS) method to project a $n$-dimensional vector representation of users' data to a lower $m$-dimensional space, and then add deliberately designed noise to satisfy differential privacy. Then, we selectively obfuscate the measurement vector under compressive sensing by adding linearly encoded noise, and provide different data reconstruction algorithms for users with different authorization levels. Extensive experimental results demonstrate that ML-DPCS yields multi-level of data utility for specific users at different authorization levels.
Matrix-free approaches for GPU acceleration of a high-order finite element hydrodynamics application using MFEM, Umpire, and RAJA
Venue: arXiv cs.MS 2021
Year: 2021

With the introduction of advanced heterogeneous computing architectures based on GPU accelerators, large-scale production codes have had to rethink their numerical algorithms and incorporate new programming models and memory management strategies in order to run efficiently on the latest supercomputers. In this work we discuss our co-design strategy to address these challenges and achieve performance and portability with MARBL, a next-generation multi-physics code in development at Lawrence Livermore National Laboratory. We present a two-fold approach, wherein new hardware is used to motivate both new algorithms and new abstraction layers, resulting in a single source application code suitable for a variety of platforms. Focusing on MARBL's ALE hydrodynamics package, we demonstrate scalability on different platforms and highlight that many of our innovations have been contributed back to open-source software libraries, such as MFEM (finite element algorithms) and RAJA (kernel abstractions).
Relative generalized Hamming weights of evaluation codes
Venue: arXiv cs.IT 2021
Year: 2021

The aim of this work is to algebraically describe the relative generalized Hamming weights of evaluation codes. We give a lower bound for these weights in terms of a footprint bound. We prove that this bound can be sharp. We compute the next-to-minimal weight of toric codes over hypersimplices of degree 1.