OpenRaiser/Intern-Atlas huggingface dataset:
Explainable Text Classification Techniques in Legal Document Review: Locating Rationales without Using Human Annotated Training Text Snippets
Venue: arXiv cs.IR 2023
Year: 2023

US corporations regularly spend millions of dollars reviewing electronically-stored documents in legal matters. Recently, attorneys apply text classification to efficiently cull massive volumes of data to identify responsive documents for use in these matters. While text classification is regularly used to reduce the discovery costs of legal matters, it also faces a perception challenge: amongst lawyers, this technology is sometimes looked upon as a "black box". Put simply, no extra information is provided for attorneys to understand why documents are classified as responsive. In recent years, explainable machine learning has emerged as an active research area. In an explainable machine learning system, predictions or decisions made by a machine learning model are human understandable. In legal 'document review' scenarios, a document is responsive, because one or more of its small text snippets are deemed responsive. In these scenarios, if these responsive snippets can be located, then attorneys could easily evaluate the model's document classification decisions - this is especially important in the field of responsible AI. Our prior research identified that predictive models created using annotated training text snippets improved the precision of a model when compared to a model created using all of a set of documents' text as training. While interesting, manually annotating training text snippets is not generally practical during a legal document review. However, small increases in precision can drastically decrease the cost of large document reviews. Automating the identification of training text snippets without human review could then make the application of training text snippet-based models a practical approach.
Machine-learning parameter tracking with partial state observation
Venue: arXiv cs.LG 2023
Year: 2023

Complex and nonlinear dynamical systems often involve parameters that change with time, accurate tracking of which is essential to tasks such as state estimation, prediction, and control. Existing machine-learning methods require full state observation of the underlying system and tacitly assume adiabatic changes in the parameter. Formulating an inverse problem and exploiting reservoir computing, we develop a model-free and fully data-driven framework to accurately track time-varying parameters from partial state observation in real time. In particular, with training data from a subset of the dynamical variables of the system for a small number of known parameter values, the framework is able to accurately predict the parameter variations in time. Low- and high-dimensional, Markovian and non-Markovian nonlinear dynamical systems are used to demonstrate the power of the machine-learning based parameter-tracking framework. Pertinent issues affecting the tracking performance are addressed.
Design Theory for Societal Digital Transformation: The Case of Digital Global Health
Venue: arXiv cs.CY 2023
Year: 2023

With societal challenges, including but not limited to human development, equity, social justice, and climate change, societal-level digital transformation (SDT) is of imminent relevance and theoretical interest. While building on local-level efforts, societal-level transformation is a nonlinear extension of the local level. Unfortunately, academic discourse on digital transformation has largely left SDT unaccounted for. Drawing on more than 25 years of intensive, interventionist research engagement with the digital transformation of public healthcare information management and delivery in more than 80 countries in the Global South, we contribute to theorizing SDT in the form of a design theory consisting of six interconnected design principles. These design principles articulate the interplay and tensions of accommodating over time increased diversity and flexibility in digital solutions, while simultaneously connecting local, national, and regional/ global efforts.
SiRA: Sparse Mixture of Low Rank Adaptation
Venue: arXiv cs.CL 2023
Year: 2023

Parameter Efficient Tuning has been an prominent approach to adapt the Large Language Model to downstream tasks. Most previous works considers adding the dense trainable parameters, where all parameters are used to adapt certain task. We found this less effective empirically using the example of LoRA that introducing more trainable parameters does not help. Motivated by this we investigate the importance of leveraging "sparse" computation and propose SiRA: sparse mixture of low rank adaption. SiRA leverages the Sparse Mixture of Expert(SMoE) to boost the performance of LoRA. Specifically it enforces the top $k$ experts routing with a capacity limit restricting the maximum number of tokens each expert can process. We propose a novel and simple expert dropout on top of gating network to reduce the over-fitting issue. Through extensive experiments, we verify SiRA performs better than LoRA and other mixture of expert approaches across different single tasks and multitask settings.
Key Factors Affecting European Reactions to AI in European Full and Flawed Democracies
Venue: arXiv cs.CY 2023
Year: 2023

This study examines the key factors that affect European reactions to artificial intelligence (AI) in the context of both full and flawed democracies in Europe. Analysing a dataset of 4,006 respondents, categorised into full democracies and flawed democracies based on the Democracy Index developed by the Economist Intelligence Unit (EIU), this research identifies crucial factors that shape European attitudes toward AI in these two types of democracies. The analysis reveals noteworthy findings. Firstly, it is observed that flawed democracies tend to exhibit higher levels of trust in government entities compared to their counterparts in full democracies. Additionally, individuals residing in flawed democracies demonstrate a more positive attitude toward AI when compared to respondents from full democracies. However, the study finds no significant difference in AI awareness between the two types of democracies, indicating a similar level of general knowledge about AI technologies among European citizens. Moreover, the study reveals that trust in AI measures, specifically "Trust AI Solution", does not significantly vary between full and flawed democracies. This suggests that despite the differences in democratic quality, both types of democracies have similar levels of confidence in AI solutions.
Quality Evaluation of Projection-Based VR Displays
Venue: arXiv cs.HC 2023
Year: 2023

We present a collection of heuristics and simple tests for evaluating the quality of a projection-based virtual reality display. A typical VR system includes numerous potential sources of error. By understanding the characteristics of a correctly working system, and the types of errors that are likely to occur, users can quickly determine if their display is inaccurate and what components may need correction.
A Simple and Powerful Framework for Stable Dynamic Network Embedding
Venue: arXiv cs.SI 2023
Year: 2023

In this paper, we address the problem of dynamic network embedding, that is, representing the nodes of a dynamic network as evolving vectors within a low-dimensional space. While the field of static network embedding is wide and established, the field of dynamic network embedding is comparatively in its infancy. We propose that a wide class of established static network embedding methods can be used to produce interpretable and powerful dynamic network embeddings when they are applied to the dilated unfolded adjacency matrix. We provide a theoretical guarantee that, regardless of embedding dimension, these unfolded methods will produce stable embeddings, meaning that nodes with identical latent behaviour will be exchangeable, regardless of their position in time or space. We additionally define a hypothesis testing framework which can be used to evaluate the quality of a dynamic network embedding by testing for planted structure in simulated networks. Using this, we demonstrate that, even in trivial cases, unstable methods are often either conservative or encode incorrect structure. In contrast, we demonstrate that our suite of stable unfolded methods are not only more interpretable but also more powerful in comparison to their unstable counterparts.
Improved Sparse Ising Optimization
Venue: arXiv cs.LG 2023
Year: 2023

Sparse Ising problems can be found in application areas such as logistics, condensed matter physics and training of deep Boltzmann networks, but can be very difficult to tackle with high efficiency and accuracy. This report presents new data demonstrating significantly higher performance on some longstanding benchmark problems with up to 20,000 variables. The data come from a new heuristic algorithm tested on the large sparse instances from the Gset benchmark suite. Relative to leading reported combinations of speed and accuracy (e.g., from Toshiba's Simulated Bifurcation Machine and Breakout Local Search), a proof-of-concept implementation reached targets 2-4 orders of magnitude faster. For two instances (G72 and G77) the new algorithm discovered a better solution than all previously reported values. Solution bitstrings confirming these two best solutions are provided. The data suggest exciting possibilities for pushing the sparse Ising performance frontier to potentially strengthen algorithm portfolios, AI toolkits and decision-making systems.
A Comparative Analysis of Machine Learning Models for Early Detection of Hospital-Acquired Infections
Venue: arXiv cs.LG 2023
Year: 2023

As more and more infection-specific machine learning models are developed and planned for clinical deployment, simultaneously running predictions from different models may provide overlapping or even conflicting information. It is important to understand the concordance and behavior of parallel models in deployment. In this study, we focus on two models for the early detection of hospital-acquired infections (HAIs): 1) the Infection Risk Index (IRI) and 2) the Ventilator-Associated Pneumonia (VAP) prediction model. The IRI model was built to predict all HAIs, whereas the VAP model identifies patients at risk of developing ventilator-associated pneumonia. These models could make important improvements in patient outcomes and hospital management of infections through early detection of infections and in turn, enable early interventions. The two models vary in terms of infection label definition, cohort selection, and prediction schema. In this work, we present a comparative analysis between the two models to characterize concordances and confusions in predicting HAIs by these models. The learnings from this study will provide important findings for how to deploy multiple concurrent disease-specific models in the future.
Local Computation Algorithms for Maximum Matching: New Lower Bounds
Venue: arXiv cs.DS 2023
Year: 2023

We study local computation algorithms (LCA) for maximum matching. An LCA does not return its output entirely, but reveals parts of it upon query. For matchings, each query is a vertex $v$; the LCA should return whether $v$ is matched -- and if so to which neighbor -- while spending a small time per query. In this paper, we prove that any LCA that computes a matching that is at most an additive of $εn$ smaller than the maximum matching in $n$-vertex graphs of maximum degree $Δ$ must take at least $Δ^{Ω(1/\varepsilon)}$ time. This comes close to the existing upper bounds that take $(Δ/ε)^{O(1/ε^2)} polylog(n)$ time. In terms of sublinear time algorithms, our techniques imply that any algorithm that estimates the size of maximum matching up to an additive error of $εn$ must take $Δ^{Ω(1/ε)}$ time. This negatively resolves a decade old open problem of the area (see Open Problem 39 of sublinear.info) on whether such estimates can be achieved in $poly(Δ/ε)$ time.
LOKE: Linked Open Knowledge Extraction for Automated Knowledge Graph Construction
Venue: arXiv cs.CL 2023
Year: 2023

While the potential of Open Information Extraction (Open IE) for Knowledge Graph Construction (KGC) may seem promising, we find that the alignment of Open IE extraction results with existing knowledge graphs to be inadequate. The advent of Large Language Models (LLMs), especially the commercially available OpenAI models, have reset expectations for what is possible with deep learning models and have created a new field called prompt engineering. We investigate the use of GPT models and prompt engineering for knowledge graph construction with the Wikidata knowledge graph to address a similar problem to Open IE, which we call Open Knowledge Extraction (OKE) using an approach we call the Linked Open Knowledge Extractor (LOKE, pronounced like "Loki"). We consider the entity linking task essential to construction of real world knowledge graphs. We merge the CaRB benchmark scoring approach with data from the TekGen dataset for the LOKE task. We then show that a well engineered prompt, paired with a naive entity linking approach (which we call LOKE-GPT), outperforms AllenAI's OpenIE 4 implementation on the OKE task, although it over-generates triples compared to the reference set due to overall triple scarcity in the TekGen set. Through an analysis of entity linkability in the CaRB dataset, as well as outputs from OpenIE 4 and LOKE-GPT, we see that LOKE-GPT and the "silver" TekGen triples show that the task is significantly different in content from OIE, if not structure. Through this analysis and a qualitative analysis of sentence extractions via all methods, we found that LOKE-GPT extractions are of high utility for the KGC task and suitable for use in semi-automated extraction settings.
On correlation bounds against polynomials
Venue: arXiv cs.CC 2023
Year: 2023

We study the fundamental challenge of exhibiting explicit functions that have small correlation with low-degree polynomials over $\mathbb{F}_{2}$. Our main contributions include: 1. In STOC 2020, CHHLZ introduced a new technique to prove correlation bounds. Using their technique they established new correlation bounds for low-degree polynomials. They conjectured that their technique generalizes to higher degree polynomials as well. We give a counterexample to their conjecture, in fact ruling out weaker parameters and showing what they prove is essentially the best possible. 2. We propose a new approach for proving correlation bounds with the central "mod functions", consisting of two steps: (I) the polynomials that maximize correlation are symmetric and (II) symmetric polynomials have small correlation. Contrary to related results in the literature, we conjecture that (I) is true. We argue this approach is not affected by existing "barrier results". 3. We prove our conjecture for quadratic polynomials. Specifically, we determine the maximum possible correlation between quadratic polynomials modulo 2 and the functions $(x_{1},\dots,x_{n})\to z^{\sum x_{i}}$ for any $z$ on the complex unit circle; and show that it is achieved by symmetric polynomials. To obtain our results we develop a new proof technique: we express correlation in terms of directional derivatives and analyze it by slowly restricting the direction. 4. We make partial progress on the conjecture for cubic polynomials, in particular proving tight correlation bounds for cubic polynomials whose degree-3 part is symmetric.
What Predicts Interpersonal Affect? Preliminary Analyses from Retrospective Evaluations
Venue: arXiv cs.HC 2023
Year: 2023

While the field of affective computing has contributed to greatly improving the seamlessness of human-robot interactions, the focus has primarily been on the emotional processing of the self, rather than the perception of the other. To address this gap, in a user study with 30 participant dyads, we collected the users' retrospective ratings of the interpersonal perception of the other interactant, after a short interaction. We made use of CORAE, a novel web-based open-source tool for COntinuous Retrospective Affect Evaluation. In this work, we analyze how these interpersonal ratings correlate with different aspects of the interaction, namely personality traits, participation balance, and sentiment analysis. Notably, we discovered that conversational imbalance has a significant effect on the retrospective ratings, among other findings. By employing these analyses and methodologies, we lay the groundwork for enhanced human-robot interactions, wherein affect is understood as a highly dynamic and context-dependent outcome of interaction history.
MoCo-Transfer: Investigating out-of-distribution contrastive learning for limited-data domains
Venue: arXiv cs.CV 2023
Year: 2023

Medical imaging data is often siloed within hospitals, limiting the amount of data available for specialized model development. With limited in-domain data, one might hope to leverage larger datasets from related domains. In this paper, we analyze the benefit of transferring self-supervised contrastive representations from moment contrast (MoCo) pretraining on out-of-distribution data to settings with limited data. We consider two X-ray datasets which image different parts of the body, and compare transferring from each other to transferring from ImageNet. We find that depending on quantity of labeled and unlabeled data, contrastive pretraining on larger out-of-distribution datasets can perform nearly as well or better than MoCo pretraining in-domain, and pretraining on related domains leads to higher performance than if one were to use the ImageNet pretrained weights. Finally, we provide a preliminary way of quantifying similarity between datasets.
A Software-Hardware Co-Optimized Toolkit for Deep Reinforcement Learning on Heterogeneous Platforms
Venue: arXiv cs.DC 2023
Year: 2023

Deep Reinforcement Learning (DRL) is vital in various AI applications. DRL algorithms comprise diverse compute kernels, which may not be simultaneously optimized using a homogeneous architecture. However, even with available heterogeneous architectures, optimizing DRL performance remains a challenge due to the complexity of hardware and programming models employed in modern data centers. To address this, we introduce PEARL, a toolkit for composing parallel DRL systems on heterogeneous platforms consisting of general-purpose processors (CPUs) and accelerators (GPUs, FPGAs). Our innovations include: 1. A general training protocol agnostic of the underlying hardware, enabling portable implementations across various processors and accelerators. 2. Incorporation of DRL-specific scheduling optimizations within the protocol, facilitating parallelized training and enhancing the overall system performance. 3. High-level API for productive development using the toolkit. 4. Automatic optimization of DRL task-to-device assignments through performance estimation, supporting various optimization metrics including throughput and power efficiency. We showcase our toolkit through experimentation with two widely used DRL algorithms, DQN and DDPG, on two diverse heterogeneous platforms. The generated implementations outperform state-of-the-art libraries for CPU-GPU platforms by throughput improvements of up to 2.1$\times$ and power efficiency improvements of up to 3.4$\times$.
HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM
Venue: arXiv cs.CL 2023
Year: 2023

Existing open-source helpfulness preference datasets do not specify what makes some responses more helpful and others less so. Models trained on these datasets can incidentally learn to model dataset artifacts (e.g. preferring longer but unhelpful responses only due to their length). To alleviate this problem, we collect HelpSteer, a multi-attribute helpfulness dataset annotated for the various aspects that make responses helpful. Specifically, our 37k-sample dataset has annotations for correctness, coherence, complexity, and verbosity in addition to overall helpfulness of responses. Training Llama 2 70B using the HelpSteer dataset with SteerLM technique produces a model that scores 7.54 on MT Bench, which is currently the highest score for open models that do not require training data from more powerful models (e.g. GPT4). We release this dataset with CC-BY-4.0 license at https://huggingface.co/datasets/nvidia/HelpSteer
TransCrimeNet: A Transformer-Based Model for Text-Based Crime Prediction in Criminal Networks
Venue: arXiv cs.CY 2023
Year: 2023

This paper presents TransCrimeNet, a novel transformer-based model for predicting future crimes in criminal networks from textual data. Criminal network analysis has become vital for law enforcement agencies to prevent crimes. However, existing graph-based methods fail to effectively incorporate crucial textual data like social media posts and interrogation transcripts that provide valuable insights into planned criminal activities. To address this limitation, we develop TransCrimeNet which leverages the representation learning capabilities of transformer models like BERT to extract features from unstructured text data. These text-derived features are fused with graph embeddings of the criminal network for accurate prediction of future crimes. Extensive experiments on real-world criminal network datasets demonstrate that TransCrimeNet outperforms previous state-of-the-art models by 12.7\% in F1 score for crime prediction. The results showcase the benefits of combining textual and graph-based features for actionable insights to disrupt criminal enterprises.
LightEMU: Hardware Assisted Fuzzing of Trusted Applications
Venue: arXiv cs.CR 2023
Year: 2023

Trusted Execution Environments (TEEs) are deployed in many CPU designs because of the confidentiality and integrity guarantees they provide. ARM TrustZone is a TEE extensively deployed on smart phones, IoT devices, and notebooks. Specifically, TrustZone is used to separate code execution and data into two worlds, normal world and secure world. However, this separation inherently prevents traditional fuzzing approaches which rely upon coverage-guided feedback and existing fuzzing research is, therefore, extremely limited. In this paper, we present a native and generic method to perform efficient and scalable feedback-driven fuzzing on Trusted Applications (TAs) using ARM CoreSight. We propose LightEMU, a novel fuzzing framework that allows us to fuzz TAs by decoupling them from relied TEE. We argue that LightEMU is a promising first-stage approach for rapidly discovering TA vulnerabilities prior to investing effort in whole system TEE evaluation precisely because the majority of publicly disclosed TrustZone bugs reside in the TA code itself. We implement LightEMU and adapt it to Teegris, Trusty, OP-TEE and QSEE and evaluate 8 real-world TAs while triggering 3 unique crashes and achieving x10 time speedup when fuzzing TAs using the state-of-the-art TrustZone fuzzing framework.