Reviewerly/exHarmony huggingface dataset:
Automatic Generation of the Draft Procuratorial Suggestions Based on an Extractive Summarization Method: BERTSLCA

The automatic generation of the draft procuratorial suggestions is to extract the description of illegal facts, administrative omission, description of laws and regulations, and other information from the case documents. Previously, the existing deep learning methods mainly focus on context-free word embeddings when addressing legal domain-speciﬁc extractive summarization tasks, which cannot get a better semantic understanding of the text and in turn leads to an adverse summarization performance. To this end, we propose a novel deep contextualized embeddings-based method BERTSLCA to conduct the extractive summarization task. The model is mainly based on the variant of BERT called BERTSUM. Firstly, the input document is fed into BERTSUM to get sentence-level embeddings. Then, we design an extracting architecture to catch the long dependency between sentences utilizing the Bi-Long Short-Term Memory (Bi-LSTM) unit, and at the end of the architecture, three cascaded convolution kernels with different sizes are designed to extract the relationships between adjacent sentences. Last, we introduce an attention mechanism to strengthen the ability to distinguish the importance of different sentences. To the best of our knowledge, this is the first work to use the pretrained language model for extractive summarization tasks in the field of Chinese judicial litigation. Experimental results on public interest litigation data and CAIL 2020 dataset all demonstrate that the proposed method achieves competitive performance.
Quantum advantage from energy measurements of many-body quantum systems

The problem of sampling outputs of quantum circuits has been proposed as a candidate for demonstrating a quantum computational advantage (sometimes referred to as quantum "supremacy"). In this work, we investigate whether quantum advantage demonstrations can be achieved for more physically-motivated sampling problems, related to measurements of physical observables. We focus on the problem of sampling the outcomes of an energy measurement, performed on a simple-to-prepare product quantum state – a problem we refer to as energy sampling. For different regimes of measurement resolution and measurement errors, we provide complexity theoretic arguments showing that the existence of efficient classical algorithms for energy sampling is unlikely. In particular, we describe a family of Hamiltonians with nearest-neighbour interactions on a 2D lattice that can be efficiently measured with high resolution using a quantum circuit of commuting gates (IQP circuit), whereas an efficient classical simulation of this process should be impossible. In this high resolution regime, which can only be achieved for Hamiltonians that can be<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow class="MJX-TeXAtom-ORD"><mml:mtext class="MJX-tex-mathit" mathvariant="italic">exponentially fast-forwarded</mml:mtext></mml:mrow></mml:math>, it is possible to use current theoretical tools tying quantum advantage statements to a polynomial-hierarchy collapse whereas for lower resolution measurements such arguments fail. Nevertheless, we show that efficient classical algorithms for low-resolution energy sampling can still be ruled out if we assume that quantum computers are strictly more powerful than classical ones. We believe our work brings a new perspective to the problem of demonstrating quantum advantage and leads to interesting new questions in Hamiltonian complexity.
Use of Generative Adversarial Networks (GAN) for Taphonomic Image Augmentation and Model Protocol for the Deep Learning Analysis of Bone Surface Modifications

Deep learning models are based on a combination of neural network architectures, optimization parameters and activation functions. All of them provide exponential combinations whose computational fitness is difficult to pinpoint. The intricate resemblance of the microscopic features that are found in bone surface modifications make their differentiation challenging, and determining a baseline combination of optimizers and activation functions for modeling seems necessary for computational economy. Here, we experiment with combinations of the most resolutive activation functions (relu, swish, and mish) and the most efficient optimizers (stochastic gradient descent (SGD) and Adam) for bone surface modification analysis. We show that despite a wide variability of outcomes, a baseline of relu–SGD is advised for raw bone surface modification data. For imbalanced samples, augmented datasets generated through generative adversarial networks are implemented, resulting in balanced accuracy and an inherent bias regarding mark replication. In summary, although baseline procedures are advised, these do not prevent to overcome Wolpert’s “no free lunch” theorem and extend it beyond model architectures.
Joint Models for Answer Verification in Question Answering Systems

This paper studies joint models for selecting correct answer sentences among the top k provided by answer sentence selection (AS2) modules, which are core components of retrievalbased Question Answering (QA) systems.Our work shows that a critical step to effectively exploiting an answer set regards modeling the interrelated information between pair of answers.For this purpose, we build a three-way multiclassifier, which decides if an answer supports, refutes, or is neutral with respect to another one.More specifically, our neural architecture integrates a state-of-the-art AS2 module with the multi-classifier, and a joint layer connecting all components.We tested our models on Wik-iQA, TREC-QA, and a real-world dataset.The results show that our models obtain the new state of the art in AS2.
Zero-shot Medical Entity Retrieval without Annotation: Learning From Rich Knowledge Graph Semantics

Medical entity retrieval is an integral component for understanding and communicating information across various health systems. Current approaches tend to work well on specific medical domains but generalize poorly to unseen sub-specialties. This is of increasing concern under a public health crisis as new medical conditions and drug treatments come to light frequently. Zero-shot retrieval is challenging due to the high degree of ambiguity and variability in medical corpora, making it difficult to build an accurate similarity measure between mentions and concepts. Medical knowledge graphs (KG), however, contain rich semantics including large numbers of synonyms as well as its curated graphical structures. To take advantage of this valuable information, we propose a suite of learning tasks designed for training efficient zero-shot entity retrieval models. Without requiring any human annotation, our knowledge graph enriched architecture significantly outperforms common zero-shot benchmarks including BM25 and Clinical BERT with 7% to 30% higher recall across multiple major medical ontologies, such as UMLS, SNOMED, and ICD-10.
An Incremental Approach to Corpus Design and Construction: Application to a Large Contemporary Saudi Corpus

Due to the rapid developments in technology and the sudden expansion of social media use, Dialect Arabic has become an important source of data that needs to be addressed when building Arabic corpora. In this paper, thirty-three Arabic corpora are surveyed to show that despite all of the developments in the literature, Saudi dialect (SD) corpora still need further expansion. This paper contributes to the literature on SD corpora by creating the largest Saudi corpus - the King Saud University Saudi Corpus (KSUSC) - with +1B total words, including +119M SD words. The KSUSC not only is the newest and largest SD corpus but is also diverse, covering 26 domains in text collected from five different sources. This paper also contributes to the literature by developing a new incremental preprocessing system that is used to create relevant lexicons that are then used to clean and normalize the collected data. This incremental system is scalable and can be adapted for different resources and dialects. Moreover, the collection process for building the KSUSC is discussed in detail, and the challenges in collecting SD text with respect to each platform are highlighted. By the end of this paper, different design criteria are proposed and used with the KSUSC to conclude that the resulting corpus can be of great benefit to researchers who are interested in integrating the corpus with their own work or using its resulting lexicons with Saudi-based NLP tasks.
GATED LINEAR NETWORKS

This paper presents a new family of backpropagation-free neural architectures, Gated Linear Networks (GLNs). What distinguishes GLNs from contemporary neural networks is the distributed and local nature of their credit assignment mechanism; each neuron directly predicts the target, forgoing the ability to learn feature representations in favor of rapid online learning. Individual neurons are able to model nonlinear functions via the use of data-dependent gating in conjunction with online convex optimization. We show that this architecture gives rise to universal learning capabilities in the limit, with effective model capacity increasing as a function of network size in a manner comparable with deep ReLU networks. Furthermore, we demonstrate that the GLN learning mechanism possesses extraordinary resilience to catastrophic forgetting, performing almost on par to an MLP with dropout and Elastic Weight Consolidation on standard benchmarks.
Deterministic algorithms for compiling quantum circuits with recurrent patterns

Current quantum processors are noisy and have limited coherence and imperfect gate implementations. On such hardware, only algorithms that are shorter than the overall coherence time can be implemented and executed successfully. A good quantum compiler must translate an input program into the most efficient equivalent of itself, getting the most out of the available hardware. In this work, we present novel deterministic algorithms for compiling recurrent quantum circuit patterns in polynomial time. In particular, such patterns appear in quantum circuits that are used to compute the ground-state properties of molecular systems using the variational quantum eigensolver method together with the RyRz heuristic wavefunction Ansätz. We show that our pattern-oriented compiling algorithms, combined with an efficient swapping strategy, produces—in general—output programs that are comparable to those obtained with state-of-the-art compilers, in terms of CNOT count and CNOT depth. In particular, our solution produces unmatched results on RyRz circuits.
Controlling nonlinear dynamical systems into arbitrary states using machine learning

Abstract Controlling nonlinear dynamical systems is a central task in many different areas of science and engineering. Chaotic systems can be stabilized (or chaotified) with small perturbations, yet existing approaches either require knowledge about the underlying system equations or large data sets as they rely on phase space methods. In this work we propose a novel and fully data driven scheme relying on machine learning (ML), which generalizes control techniques of chaotic systems without requiring a mathematical model for its dynamics. Exploiting recently developed ML-based prediction capabilities, we demonstrate that nonlinear systems can be forced to stay in arbitrary dynamical target states coming from any initial state. We outline and validate our approach using the examples of the Lorenz and the Rössler system and show how these systems can very accurately be brought not only to periodic, but even to intermittent and different chaotic behavior. Having this highly flexible control scheme with little demands on the amount of required data on hand, we briefly discuss possible applications ranging from engineering to medicine.
Russian students about learning under the covid-19 pandemic: Resources, opportunities and assessment of the distance learning

The covid-19 pandemic has made the whole world face specific challenges that required urgent decisions on how to organize the work of social institutions, including education, in this new reality. Since March 2020, national education systems all over the world have been adapting their activities to the quarantine measures and have been switching to distance learning. In varying degrees, these changes affected all levels of education - from preschool to higher and postgraduate education. However, the tasks to be solved differed by the level of education. The article presents the results of the comparative analysis of the activities of the Russian higher-educational institutions based on their students estimates - during the forced distance learning in the spring of 2020 and during the planned switch to such a format in the fall of 2020. The article is based on the surveys of the Russian students conducted in June 2020 (N=31423) and February 2021 (N=32358). The study was conducted to test the hypotheses that negative estimates of distance learning and rejection of such a format by many students in the spring of 2020 changed in the fall of 2020. The results of two surveys show that Russian universities have passed through the stage of digital transformation which affected both the resources for organizing the educational process and the competences of the teaching staff. The experience of work in the difficult and unpredictable conditions of the pandemic changed the public opinion from the negative perception of distance learning to understanding and accepting its positive features for the education system and its actors. The introduction of digital technologies is an objective and inevitable process in the development of the educational sphere and the country.
Changes in European Solidarity Before and During COVID-19: Evidence from a Large Crowd- and Expert-Annotated Twitter Dataset

We introduce the well-established social scientific concept of social solidarity and its contestation, anti-solidarity, as a new problem setting to supervised machine learning in NLP to assess how European solidarity discourses changed before and after the COVID-19 outbreak was declared a global pandemic.To this end, we annotate 2.3k English and German tweets for (anti-)solidarity expressions, utilizing multiple human annotators and two annotation approaches (experts vs. crowds).We use these annotations to train a BERT model with multiple data augmentation strategies.Our augmented BERT model that combines both expert and crowd annotations outperforms the baseline BERT classifier trained with expert annotations only by over 25 points, from 58% macro-F1 to almost 85%.We use this highquality model to automatically label over 270k tweets between September 2019 and December 2020.We then assess the automatically labeled data for how statements related to European (anti-)solidarity discourses developed over time and in relation to one another, before and during the COVID-19 crisis.Our results show that solidarity became increasingly salient and contested during the crisis.While the number of solidarity tweets remained on a higher level and dominated the discourse in the scrutinized time frame, anti-solidarity tweets initially spiked, then decreased to (almost) pre-COVID-19 values before rising to a stable higher level until the end of 2020.
Image orientation detection by ensembles of Stochastic CNNs

In this paper we deal with the problem of accurately and automatically detecting the orientation of general images, for instance, of holiday snapshots.Detecting image orientation is an easy task for a human being but can be a long and tedious activity during processing and management of digital photos.Several attempts have been made in the design of systems for automated displaying images in their correct orientation, however, this is still an open problem.In this work we exploit the power of deep learning proposing a transfer learning approach that adjusts pre-trained convolutional neural networks to this classification task.We create ensembles of different Convolutional Neural Network models designed by randomly changing the activation functions in all the activation layers of a given network.Along with several known activation functions we also include the novel Soft Learnable activation function in the ''random set''.Our resulting ensembles have been extensively evaluated on more than 45,000 images taken from four different public datasets, showing a remarkable performance improvement with respect to other state-of-the-art approaches.
Image Inpainting Using Wasserstein Generative Adversarial Imputation Network

Image inpainting is one of the important tasks in computer vision which focuses on the reconstruction of missing regions in an image. The aim of this paper is to introduce an image inpainting model based on Wasserstein Generative Adversarial Imputation Network. The generator network of the model uses building blocks of convolutional layers with different dilation rates, together with skip connections that help the model reproduce fine details of the output. This combination yields a universal imputation model that is able to handle various scenarios of missingness with sufficient quality. To show this experimentally, the model is simultaneously trained to deal with three scenarios given by missing pixels at random, missing various smaller square regions, and one missing square placed in the center of the image. It turns out that our model achieves high-quality inpainting results on all scenarios. Performance is evaluated using peak signal-to-noise ratio and structural similarity index on two real-world benchmark datasets, CelebA faces and Paris StreetView. The results of our model are compared to biharmonic imputation and to some of the other state-of-the-art image inpainting methods.
Distributed GM-CPHD Filter Based on Generalized Inverse Covariance Intersection

In this paper, we propose a distributed Gaussian mixture cardinalized probability hypothesis density (GM-CPHD) filter based on generalized inverse covariance intersection that fuses multiple node information effectively for multi-target tracking applications.Covariance intersection (CI) is a well-known fusion method that produces a conservative estimate of the joint covariance regardless of the actual correlation between the different nodes.Inverse covariance intersection (ICI) is the updated version to obtain fusion results that guarantee consistency and less conservative than CI.However, the ICI is not extended to multi-sensor multi-target tracking system yet.Since the ICI formula can be re-structured as naïve fusion with covariance inflation in Gaussian pdf, this method was applied to the GM-CPHD with generalization.The formula for random finite set (RFS) fusion was derived in the same way as the conventional generalized covariance intersection (GCI) based fusion.The simulation results for multi-target tracking show that the proposed algorithm has smaller optimal sub-pattern assignment (OSPA) errors than naïve fusion and the GCI-based fusions.
Resilient Multi-Agent Reinforcement Learning with Adversarial Value Decomposition

We focus on resilience in cooperative multi-agent systems, where agents can change their behavior due to udpates or failures of hardware and software components. Current state-of-the-art approaches to cooperative multi-agent reinforcement learning (MARL) have either focused on idealized settings without any changes or on very specialized scenarios, where the number of changing agents is fixed, e.g., in extreme cases with only one productive agent. Therefore, we propose Resilient Adversarial value Decomposition with Antagonist-Ratios (RADAR). RADAR offers a value decomposition scheme to train competing teams of varying size for improved resilience against arbitrary agent changes. We evaluate RADAR in two cooperative multi-agent domains and show that RADAR achieves better worst case performance w.r.t. arbitrary agent changes than state-of-the-art MARL.
Depth Privileged Object Detection in Indoor Scenes via Deformation Hallucination

RGB-D object detection has achieved significant advance, because depth provides complementary geometric information to RGB images. Considering depth images are unavailable in some scenarios, we focus on depth privileged object detection in indoor scenes, where the depth images are only available in the training phase. Under this setting, one prevalent research line is modality hallucination, in which depth image and depth feature are the common choices for hallucinating. In contrast, we choose to hallucinate depth deformation, which is explicit geometric information and efficient to hallucinate. Specifically, we employ the deformable convolution layer with augmented offsets as our deformation module and regard the offsets as geometric deformation, because the offsets enable flexibly sampling over the object and transforming to a canonical shape for ease of detection. In addition, we design a quality-based mechanism to avoid negative transfer of depth deformation. Experimental results and analyses on NYUDv2 and SUN RGB-D demonstrate the effectiveness of our method against the state-of-the-art methods for depth privileged object detection.
ParallelFusion

Mobile GPUs are extremely under-utilized for DNN computations across different mobile deep learning frameworks and multiple DNNs with various complexities. We explore the feasibility of batching and it improves the throughput by up to 35%. However, real-time applications in mobile have a limited amount of requests to get a benefit from batching. To tackle the challenge, we present ParallelFusion technique that enables concurrent execution of heterogeneous operators to further utilize the mobile GPU. We implemented ParallelFusion over the MNN framework and evaluated on 6 state-of-the-art DNNs. Our evaluation shows that Parallel Fusion achieves up to 195% to 218% throughput with fused execution of 2 and 3 operators compared to single DNN inference.
Learning Flexibly Distributional Representation for Low-quality 3D Face Recognition

Due to the superiority of using geometric information, 3D Face Recognition (FR) has achieved great successes. Existing methods focus on high-quality 3D FR which is unpractical in real scenarios. Low-quality 3D FR is a more realistic scenario but the low-quality data are born with heavy noises. Therefore, exploring noise-robust low-quality 3D FR methods becomes an urgent and challenging problem. To solve this issue, in this paper, we propose to learn flexibly distributional representation for low-quality 3D FR. Firstly, we introduce the distributional representation for low-quality 3D faces due to that it can weaken the impact of noises. Generally, the distributional representation of a given 3D face is restricted to a specific distribution such as Gaussian distribution. However, the specific distribution may be not up to describing the complex low-quality face. Therefore, we propose to transform this specific distribution to a flexible one via Continuous Normalizing Flow (CNF), which can get rid of the form limitation. This kind of flexible distribution can approximate the latent distribution of the given noisy face more accurately, which further improves accuracy of low-quality 3D FR. Comprehensive experiments show that our proposed method improves both low-quality and cross-quality 3D FR performances on low-quality benchmarks. Furthermore, the improvements are more remarkable on low-quality 3D faces when the intensity of noise increases which indicate the robustness