OpenRaiser/Intern-Atlas huggingface dataset:
Knowledge is Power: Understanding Causality Makes Legal judgment Prediction Models More Generalizable and Robust
Venue: arXiv cs.CL 2023
Year: 2023

Legal Judgment Prediction (LJP), aiming to predict a judgment based on fact descriptions according to rule of law, serves as legal assistance to mitigate the great work burden of limited legal practitioners. Most existing methods apply various large-scale pre-trained language models (PLMs) finetuned in LJP tasks to obtain consistent improvements. However, we discover the fact that the state-of-the-art (SOTA) model makes judgment predictions according to irrelevant (or non-casual) information. The violation of rule of law not only weakens the robustness and generalization ability of models but also results in severe social problems like discrimination. In this paper, we use causal structural models (SCMs) to theoretically analyze how LJP models learn to make decisions and why they can succeed in passing the traditional testing paradigm without learning causality. According to our analysis, we provide two solutions intervening on data and model by causality, respectively. In detail, we first distinguish non-causal information by applying the open information extraction (OIE) technique. Then, we propose a method named the Causal Information Enhanced SAmpling Method (CIESAM) to eliminate the non-causal information from data. To validate our theoretical analysis, we further propose another method using our proposed Causality-Aware Self-Attention Mechanism (CASAM) to guide the model to learn the underlying causality knowledge in legal texts. The confidence of CASAM in learning causal information is higher than that of CIESAM. The extensive experimental results show that both our proposed methods achieve state-of-the-art (SOTA) performance on three commonly used legal-specific datasets. The stronger performance of CASAM further demonstrates that causality is the key to the robustness and generalization ability of models.
Online Nash Welfare Maximization Without Predictions
Venue: arXiv cs.DS 2023
Year: 2023

The maximization of Nash welfare, which equals the geometric mean of agents' utilities, is widely studied because it balances efficiency and fairness in resource allocation problems. Banerjee, Gkatzelis, Gorokh, and Jin (2022) recently introduced the model of online Nash welfare maximization for $T$ divisible items and $N$ agents with additive utilities with predictions of each agent's utility for receiving all items. They gave online algorithms whose competitive ratios are logarithmic. We initiate the study of online Nash welfare maximization \emph{without predictions}, assuming either that the agents' utilities for receiving all items differ by a bounded ratio, or that their utilities for the Nash welfare maximizing allocation differ by a bounded ratio. We design online algorithms whose competitive ratios only depend on the logarithms of the aforementioned ratios of agents' utilities and the number of agents.
Confidence-Ranked Reconstruction of Census Microdata from Published Statistics
Venue: arXiv cs.CY 2023
Year: 2023

A reconstruction attack on a private dataset $D$ takes as input some publicly accessible information about the dataset and produces a list of candidate elements of $D$. We introduce a new class of data reconstruction attacks based on randomized methods for non-convex optimization. We empirically demonstrate that our attacks can not only reconstruct full rows of $D$ from aggregate query statistics $Q(D)\in \mathbb{R}^m$, but can do so in a way that reliably ranks reconstructed rows by their odds of appearing in the private data, providing a signature that could be used for prioritizing reconstructed rows for further actions such as identify theft or hate crime. We also design a sequence of baselines for evaluating reconstruction attacks. Our attacks significantly outperform those that are based only on access to a public distribution or population from which the private dataset $D$ was sampled, demonstrating that they are exploiting information in the aggregate statistics $Q(D)$, and not simply the overall structure of the distribution. In other words, the queries $Q(D)$ are permitting reconstruction of elements of this dataset, not the distribution from which $D$ was drawn. These findings are established both on 2010 U.S. decennial Census data and queries and Census-derived American Community Survey datasets. Taken together, our methods and experiments illustrate the risks in releasing numerically precise aggregate statistics of a large dataset, and provide further motivation for the careful application of provably private techniques such as differential privacy.
Zero-Shot Classification by Logical Reasoning on Natural Language Explanations
Venue: arXiv cs.CL 2023
Year: 2023

Humans can classify data of an unseen category by reasoning on its language explanations. This ability is owing to the compositional nature of language: we can combine previously seen attributes to describe the new category. For example, we might describe a sage thrasher as "it has a slim straight relatively short bill, yellow eyes and a long tail", so that others can use their knowledge of attributes "slim straight relatively short bill", "yellow eyes" and "long tail" to recognize a sage thrasher. Inspired by this observation, in this work we tackle zero-shot classification task by logically parsing and reasoning on natural language expla-nations. To this end, we propose the framework CLORE (Classification by LOgical Reasoning on Explanations). While previous methods usually regard textual information as implicit features, CLORE parses explanations into logical structures and then explicitly reasons along thess structures on the input to produce a classification score. Experimental results on explanation-based zero-shot classification benchmarks demonstrate that CLORE is superior to baselines, which we further show mainly comes from higher scores on tasks requiring more logical reasoning. We also demonstrate that our framework can be extended to zero-shot classification on visual modality. Alongside classification decisions, CLORE can provide the logical parsing and reasoning process as a clear form of rationale. Through empirical analysis we demonstrate that CLORE is also less affected by linguistic biases than baselines.
RITA: Boost Driving Simulators with Realistic Interactive Traffic Flow
Venue: arXiv cs.AI 2023
Year: 2023

High-quality traffic flow generation is the core module in building simulators for autonomous driving. However, the majority of available simulators are incapable of replicating traffic patterns that accurately reflect the various features of real-world data while also simulating human-like reactive responses to the tested autopilot driving strategies. Taking one step forward to addressing such a problem, we propose Realistic Interactive TrAffic flow (RITA) as an integrated component of existing driving simulators to provide high-quality traffic flow for the evaluation and optimization of the tested driving strategies. RITA is developed with consideration of three key features, i.e., fidelity, diversity, and controllability, and consists of two core modules called RITABackend and RITAKit. RITABackend is built to support vehicle-wise control and provide traffic generation models from real-world datasets, while RITAKit is developed with easy-to-use interfaces for controllable traffic generation via RITABackend. We demonstrate RITA's capacity to create diversified and high-fidelity traffic simulations in several highly interactive highway scenarios. The experimental findings demonstrate that our produced RITA traffic flows exhibit all three key features, hence enhancing the completeness of driving strategy evaluation. Moreover, we showcase the possibility for further improvement of baseline strategies through online fine-tuning with RITA traffic flows.
Knowledge Graph Embedding: A Survey from the Perspective of Representation Spaces
Venue: arXiv cs.LG 2023
Year: 2023

Knowledge graph embedding (KGE) is an increasingly popular technique that aims to represent entities and relations of knowledge graphs into low-dimensional semantic spaces for a wide spectrum of applications such as link prediction, knowledge reasoning and knowledge completion. In this paper, we provide a systematic review of existing KGE techniques based on representation spaces. Particularly, we build a fine-grained classification to categorise the models based on three mathematical perspectives of the representation spaces: (1) Algebraic perspective, (2) Geometric perspective, and (3) Analytical perspective. We introduce the rigorous definitions of fundamental mathematical spaces before diving into KGE models and their mathematical properties. We further discuss different KGE methods over the three categories, as well as summarise how spatial advantages work over different embedding needs. By collating the experimental results from downstream tasks, we also explore the advantages of mathematical space in different scenarios and the reasons behind them. We further state some promising research directions from a representation space perspective, with which we hope to inspire researchers to design their KGE models as well as their related applications with more consideration of their mathematical space properties.
Spatiotemporal forecasting of vertical track alignment with exogenous factors
Venue: arXiv cs.LG 2023
Year: 2023

To ensure the safety of railroad operations, it is important to monitor and forecast track geometry irregularities. A higher safety requires forecasting with higher spatiotemporal frequencies, which in turn requires capturing spatial correlations. Additionally, track geometry irregularities are influenced by multiple exogenous factors. In this study, a method is proposed to forecast one type of track geometry irregularity, vertical alignment, by incorporating spatial and exogenous factor calculations. The proposed method embeds exogenous factors and captures spatiotemporal correlations using a convolutional long short-term memory. The proposed method is also experimentally compared with other methods in terms of the forecasting performance. Additionally, an ablation study on exogenous factors is conducted to examine their individual contributions to the forecasting performance. The results reveal that spatial calculations and maintenance record data improve the forecasting of vertical alignment.
On a Network Centrality Maximization Game
Venue: arXiv cs.SI 2023
Year: 2023

We study a network formation game where $n$ players, identified with the nodes of a directed graph to be formed, choose where to wire their outgoing links in order to maximize their PageRank centrality. Specifically, the action of every player $i$ consists in the wiring of a predetermined number $d_i$ of directed out-links, and her utility is her own PageRank centrality in the network resulting from the actions of all players. We show that this is a potential game and that the best response correspondence always exhibits a local structure in that it is never convenient for a node $i$ to link to other nodes that are at incoming distance more than $d_i $ from her. We then study the equilibria of this game determining necessary conditions for a graph to be a (strict, recurrent) Nash equilibrium. Moreover, in the homogeneous case, where players all have the same number $d$ of out-links, we characterize the structure of the potential maximizing equilibria and, in the special cases $ d=1 $ and $ d=2 $, we provide a complete classification of the set of (strict, recurrent) Nash equilibria. Our analysis shows in particular that the considered formation mechanism leads to the emergence of undirected and disconnected or loosely connected networks.
ATCO2 corpus: A Large-Scale Dataset for Research on Automatic Speech Recognition and Natural Language Understanding of Air Traffic Control Communications
Venue: arXiv cs.CL 2023
Year: 2023

Personal assistants, automatic speech recognizers and dialogue understanding systems are becoming more critical in our interconnected digital world. A clear example is air traffic control (ATC) communications. ATC aims at guiding aircraft and controlling the airspace in a safe and optimal manner. These voice-based dialogues are carried between an air traffic controller (ATCO) and pilots via very-high frequency radio channels. In order to incorporate these novel technologies into ATC (low-resource domain), large-scale annotated datasets are required to develop the data-driven AI systems. Two examples are automatic speech recognition (ASR) and natural language understanding (NLU). In this paper, we introduce the ATCO2 corpus, a dataset that aims at fostering research on the challenging ATC field, which has lagged behind due to lack of annotated data. The ATCO2 corpus covers 1) data collection and pre-processing, 2) pseudo-annotations of speech data, and 3) extraction of ATC-related named entities. The ATCO2 corpus is split into three subsets. 1) ATCO2-test-set corpus contains 4 hours of ATC speech with manual transcripts and a subset with gold annotations for named-entity recognition (callsign, command, value). 2) The ATCO2-PL-set corpus consists of 5281 hours of unlabeled ATC data enriched with automatic transcripts from an in-domain speech recognizer, contextual information, speaker turn information, signal-to-noise ratio estimate and English language detection score per sample. Both available for purchase through ELDA at http://catalog.elra.info/en-us/repository/browse/ELRA-S0484. 3) The ATCO2-test-set-1h corpus is a one-hour subset from the original test set corpus, that we are offering for free at https://www.atco2.org/data. We expect the ATCO2 corpus will foster research on robust ASR and NLU not only in the field of ATC communications but also in the general research community.
Why we couldn't prove SETH hardness of the Closest Vector Problem for even norms!
Venue: arXiv cs.CC 2023
Year: 2023

Recent work [BGS17,ABGS19] has shown SETH hardness of CVP in the $\ell_p$ norm for any $p$ that is not an even integer. This result was shown by giving a Karp reduction from $k$-SAT on $n$ variables to CVP on a lattice of rank $n$. In this work, we show a barrier towards proving a similar result for CVP in the $\ell_p$ norm where $p$ is an even integer. We show that for any $c>0$, if for every $k > 0$, there exists an efficient reduction that maps a $k$-SAT instance on $n$ variables to a CVP instance for a lattice of rank at most $n^{c}$ in the Euclidean norm, then $\mathsf{coNP} \subset \mathsf{NP/Poly}$. We prove a similar result for CVP for all even norms under a mild additional promise that the ratio of the distance of the target from the lattice and the shortest non-zero vector in the lattice is bounded by $exp(n^{O(1)})$. Furthermore, we show that for any $c> 0$, and any even integer $p$, if for every $k > 0$, there exists an efficient reduction that maps a $k$-SAT instance on $n$ variables to a $SVP_p$ instance for a lattice of rank at most $n^{c}$, then $\mathsf{coNP} \subset \mathsf{NP/Poly}$. The result for SVP does not require any additional promise. While prior results have indicated that lattice problems in the $\ell_2$ norm (Euclidean norm) are easier than lattice problems in other norms, this is the first result that shows a separation between these problems. We achieve this by using a result by Dell and van Melkebeek [JACM, 2014] on the impossibility of the existence of a reduction that compresses an arbitrary $k$-SAT instance into a string of length $\mathcal{O}(n^{k-ε})$ for any $ε>0$. In addition to CVP, we also show that the same result holds for the Subset-Sum problem using similar techniques.
Asymptotically Optimal Stochastic Lossy Coding of Markov Sources
Venue: arXiv cs.IT 2023
Year: 2023

An effective 'on-the-fly' mechanism for stochastic lossy coding of Markov sources using string matching techniques is proposed in this paper. Earlier work has shown that the rate-distortion bound can be asymptotically achieved by a 'natural type selection' (NTS) mechanism which iteratively encodes asymptotically long source strings (from an unknown source distribution P) and regenerates the codebook according to a maximum likelihood distribution framework, after observing a set of K codewords to 'd-match' (i.e., satisfy the distortion constraint for) a respective set of K source words. This result was later generalized for sources with memory under the assumption that the source words must contain a sequence of asymptotic-length vectors (or super-symbols) over the source super-alphabet, i.e., the source is considered a vector source. However, the earlier result suffers from a significant practical flaw, more specifically, it requires expanding the super-symbols (and correspondingly the super-alphabet) lengths to infinity in order to achieve the rate-distortion bound, even for finite memory sources, e.g., Markov sources. This implies that the complexity of the NTS iteration will explode beyond any practical capabilities, thus compromising the promise of the NTS algorithm in practical scenarios for sources with memory. This work describes a considerably more efficient and tractable mechanism to achieve asymptotically optimal performance given a prescribed memory constraint, within a practical framework tailored to Markov sources. More specifically, the algorithm finds asymptotically the optimal codebook reproduction distribution, within a constrained set of distributions having Markov property with a prescribed order, that achieves the minimum per letter coding rate while maintaining a specified distortion level.
SUPRA: Superpixel Guided Loss for Improved Multi-modal Segmentation in Endoscopy
Venue: arXiv cs.CV 2023
Year: 2023

Domain shift is a well-known problem in the medical imaging community. In particular, for endoscopic image analysis where the data can have different modalities the performance of deep learning (DL) methods gets adversely affected. In other words, methods developed on one modality cannot be used for a different modality. However, in real clinical settings, endoscopists switch between modalities for better mucosal visualisation. In this paper, we explore the domain generalisation technique to enable DL methods to be used in such scenarios. To this extend, we propose to use super pixels generated with Simple Linear Iterative Clustering (SLIC) which we refer to as "SUPRA" for SUPeRpixel Augmented method. SUPRA first generates a preliminary segmentation mask making use of our new loss "SLICLoss" that encourages both an accurate and color-consistent segmentation. We demonstrate that SLICLoss when combined with Binary Cross Entropy loss (BCE) can improve the model's generalisability with data that presents significant domain shift. We validate this novel compound loss on a vanilla U-Net using the EndoUDA dataset, which contains images for Barret's Esophagus and polyps from two modalities. We show that our method yields an improvement of nearly 20% in the target domain set compared to the baseline.
Errorless Robust JPEG Steganography using Outputs of JPEG Coders
Venue: arXiv cs.MM 2023
Year: 2023

Robust steganography is a technique of hiding secret messages in images so that the message can be recovered after additional image processing. One of the most popular processing operations is JPEG recompression. Unfortunately, most of today's steganographic methods addressing this issue only provide a probabilistic guarantee of recovering the secret and are consequently not errorless. That is unacceptable since even a single unexpected change can make the whole message unreadable if it is encrypted. We propose to create a robust set of DCT coefficients by inspecting their behavior during recompression, which requires access to the targeted JPEG compressor. This is done by dividing the DCT coefficients into 64 non-overlapping lattices because one embedding change can potentially affect many other coefficients from the same DCT block during recompression. The robustness is then combined with standard steganographic costs creating a lattice embedding scheme robust against JPEG recompression. Through experiments, we show that the size of the robust set and the scheme's security depends on the ordering of lattices during embedding. We verify the validity of the proposed method with three typical JPEG compressors and the {\it Slack} instant messaging application. We benchmark its security for various embedding payloads, three different ways of ordering the lattices, and a range of Quality Factors. Finally, this method is errorless by construction, meaning the embedded message will always be readable.
Global, and Local Optimization Beamforming for Broadband Sources
Venue: arXiv cs.SD 2023
Year: 2023

This paper presents an alternative energy function for Global Optimization (GO) beamforming, tailored to acoustic broadband sources. Given, that properties such as the source location, multipole rotation, or flow conditions are parameterized over the frequency, a CSM-fitting can be performed for all frequencies at once. A numerical analysis shows that the nonlinear energy function for the standard GO problem is equivalent to the source's Point Spread Function (PSF) and contains local minima at the grating- and side lobes' locations. The energy function is improved with the proposed broadband energy, as it averages the PSF. Further, it simplifies the process of identifying sources and reconstructing their spectra from the results. The paper shows that the method is superior on synthetic monopoles compared to standard GO and CLEAN-SC. For real-world data the results of the proposed method and CLEAN-SC are similar, and outperform standard GO. The main difference is that source assumption violations cause noisy maps for CLEAN-SC and cause wrong spectral estimations of the proposed method. By using reasonable initial values, the GO problem reduces to a Local Optimization problem with similar results. Further, the proposed method is able to identify synthetic multipoles with different pole amplitudes and unknown pole rotations.
QuadConv: Quadrature-Based Convolutions with Applications to Non-Uniform PDE Data Compression
Venue: arXiv cs.LG 2023
Year: 2023

We present a new convolution layer for deep learning architectures which we call QuadConv -- an approximation to continuous convolution via quadrature. Our operator is developed explicitly for use on non-uniform, mesh-based data, and accomplishes this by learning a continuous kernel that can be sampled at arbitrary locations. Moreover, the construction of our operator admits an efficient implementation which we detail and construct. As an experimental validation of our operator, we consider the task of compressing partial differential equation (PDE) simulation data from fixed meshes. We show that QuadConv can match the performance of standard discrete convolutions on uniform grid data by comparing a QuadConv autoencoder (QCAE) to a standard convolutional autoencoder (CAE). Further, we show that the QCAE can maintain this accuracy even on non-uniform data. In both cases, QuadConv also outperforms alternative unstructured convolution methods such as graph convolution.