OpenRaiser/Intern-Atlas huggingface dataset:
Unveiling the Digital Fingerprints: Analysis of Internet attacks based on website fingerprints
Venue: arXiv cs.CR 2024
Year: 2024

Parallel to our physical activities our virtual presence also leaves behind our unique digital fingerprints, while navigating on the Internet. These digital fingerprints have the potential to unveil users' activities encompassing browsing history, utilized applications, and even devices employed during these engagements. Many Internet users tend to use web browsers that provide the highest privacy protection and anonymization such as Tor. The success of such privacy protection depends on the Tor feature to anonymize end-user IP addresses and other metadata that constructs the website fingerprint. In this paper, we show that using the newest machine learning algorithms an attacker can deanonymize Tor traffic by applying such techniques. In our experimental framework, we establish a baseline and comparative reference point using a publicly available dataset from Universidad Del Cauca, Colombia. We capture network packets across 11 days, while users navigate specific web pages, recording data in .pcapng format through the Wireshark network capture tool. Excluding extraneous packets, we employ various machine learning algorithms in our analysis. The results show that the Gradient Boosting Machine algorithm delivers the best outcomes in binary classification, achieving an accuracy of 0.8363. In the realm of multi-class classification, the Random Forest algorithm attains an accuracy of 0.6297.
Sirius: Contextual Sparsity with Correction for Efficient LLMs
Venue: arXiv cs.CL 2024
Year: 2024

With the blossom of large language models (LLMs), inference efficiency becomes increasingly important. Various approximation methods are proposed to reduce the cost at inference time. Contextual Sparsity (CS) is appealing for its training-free nature and its ability to reach a higher compression ratio seemingly without quality degradation. However, after a comprehensive evaluation of contextual sparsity methods on various complex generation tasks, we find that although CS succeeds in prompt-understanding tasks, CS significantly degrades the model performance for reasoning, deduction, and knowledge-based tasks. Despite the gap in end-to-end accuracy, we observed that sparse models often share general problem-solving logic and require only a few token corrections to recover the original model performance. This paper introduces Sirius, an efficient correction mechanism, which significantly recovers CS models quality on reasoning tasks while maintaining its efficiency gain. Sirius is evaluated on 6 models with 8 difficult generation tasks in reasoning, math, and coding and shows consistent effectiveness and efficiency. Also, we carefully develop a system implementation for Sirius and show that Sirius achieves roughly 20% reduction in latency for 8B model on-chip and 35% reduction for 70B model offloading. We open-source our implementation of Sirius at https://github.com/Infini-AI-Lab/Sirius.git.
Constant congestion linkages in polynomially strong digraphs in polynomial time
Venue: arXiv cs.DS 2024
Year: 2024

Given integers $k,c > 0$, we say that a digraph $D$ is $(k,c)$-linked if for every pair of ordered sets $\{s_1, \ldots, s_k\}$ and $\{t_1, \ldots, t_k\}$ of vertices of $D$, there are $P_1, \ldots, P_k$ such that for $i \in [k]$ each $P_i$ is a path from $s_i$ to $t_i$ and every vertex of $D$ appears in at most $c$ of those paths. Thomassen [Combinatorica, 1991] showed that for every fixed $k \geq 2$ there is no integer $p$ such that every $p$-strong digraph is $(k,1)$-linked. Edwards et al. [ESA, 2017] showed that every digraph $D$ with directed treewidth at least some function $f(k)$ contains a large bramble of congestion $2$ and that every $(36k^3 + 2k)$-strong digraph containing a bramble of congestion $2$ and size roughly $188k^3$ is $(k,2)$-linked. Since the directed treewidth of a digraph has to be at least its strong connectivity, this implies that there is a function $L(k)$ such that every $L(k)$-strong digraph is $(k,2)$-linked. This result was improved by Campos et al. [ESA, 2023], who showed that any $k$-strong digraph containing a bramble of size at least $2k(c\cdot k -c + 2) + c(k-1)$ and congestion $c$ is $(k,c)$-linked. Regarding the bramble, although the given bound on $f(k)$ is very large, Masařík et al. [SIDMA, 2022] showed that directed treewidth $\mathcal{O}(k^{48}\log^{13} k)$ suffices if the congestion is relaxed to $8$. We first show how to drop the dependence on $c$, for even $c$, on the size of the bramble that is needed in the work of Campos et al. [ESA, 2023]. Then, by making two local changes in the proof of Masařík et al. [SIDMA, 2022] we show how to build in polynomial time a bramble of size $k$ and congestion $8$ assuming that a large obstruction to directed treewidth (namely, a path system) is given. Applying these results, we show that there is a polynomial function $g(k)$ such that every $g(k)$-strong digraph is $(k,8)$-linked.
CACER: Clinical Concept Annotations for Cancer Events and Relations
Venue: arXiv cs.CL 2024
Year: 2024

Clinical notes contain unstructured representations of patient histories, including the relationships between medical problems and prescription drugs. To investigate the relationship between cancer drugs and their associated symptom burden, we extract structured, semantic representations of medical problem and drug information from the clinical narratives of oncology notes. We present Clinical Concept Annotations for Cancer Events and Relations (CACER), a novel corpus with fine-grained annotations for over 48,000 medical problems and drug events and 10,000 drug-problem and problem-problem relations. Leveraging CACER, we develop and evaluate transformer-based information extraction (IE) models such as BERT, Flan-T5, Llama3, and GPT-4 using fine-tuning and in-context learning (ICL). In event extraction, the fine-tuned BERT and Llama3 models achieved the highest performance at 88.2-88.0 F1, which is comparable to the inter-annotator agreement (IAA) of 88.4 F1. In relation extraction, the fine-tuned BERT, Flan-T5, and Llama3 achieved the highest performance at 61.8-65.3 F1. GPT-4 with ICL achieved the worst performance across both tasks. The fine-tuned models significantly outperformed GPT-4 in ICL, highlighting the importance of annotated training data and model optimization. Furthermore, the BERT models performed similarly to Llama3. For our task, LLMs offer no performance advantage over the smaller BERT models. The results emphasize the need for annotated training data to optimize models. Multiple fine-tuned transformer models achieved performance comparable to IAA for several extraction tasks.
Data-Efficient Generation for Dataset Distillation
Venue: arXiv cs.CV 2024
Year: 2024

While deep learning techniques have proven successful in image-related tasks, the exponentially increased data storage and computation costs become a significant challenge. Dataset distillation addresses these challenges by synthesizing only a few images for each class that encapsulate all essential information. Most current methods focus on matching. The problems lie in the synthetic images not being human-readable and the dataset performance being insufficient for downstream learning tasks. Moreover, the distillation time can quickly get out of bounds when the number of synthetic images per class increases even slightly. To address this, we train a class conditional latent diffusion model capable of generating realistic synthetic images with labels. The sampling time can be reduced to several tens of images per seconds. We demonstrate that models can be effectively trained using only a small set of synthetic images and evaluated on a large real test set. Our approach achieved rank \(1\) in The First Dataset Distillation Challenge at ECCV 2024 on the CIFAR100 and TinyImageNet datasets.
Automating Robot Failure Recovery Using Vision-Language Models With Optimized Prompts
Venue: arXiv cs.RO 2024
Year: 2024

Current robot autonomy struggles to operate beyond the assumed Operational Design Domain (ODD), the specific set of conditions and environments in which the system is designed to function, while the real-world is rife with uncertainties that may lead to failures. Automating recovery remains a significant challenge. Traditional methods often rely on human intervention to manually address failures or require exhaustive enumeration of failure cases and the design of specific recovery policies for each scenario, both of which are labor-intensive. Foundational Vision-Language Models (VLMs), which demonstrate remarkable common-sense generalization and reasoning capabilities, have broader, potentially unbounded ODDs. However, limitations in spatial reasoning continue to be a common challenge for many VLMs when applied to robot control and motion-level error recovery. In this paper, we investigate how optimizing visual and text prompts can enhance the spatial reasoning of VLMs, enabling them to function effectively as black-box controllers for both motion-level position correction and task-level recovery from unknown failures. Specifically, the optimizations include identifying key visual elements in visual prompts, highlighting these elements in text prompts for querying, and decomposing the reasoning process for failure detection and control generation. In experiments, prompt optimizations significantly outperform pre-trained Vision-Language-Action Models in correcting motion-level position errors and improve accuracy by 65.78% compared to VLMs with unoptimized prompts. Additionally, for task-level failures, optimized prompts enhanced the success rate by 5.8%, 5.8%, and 7.5% in VLMs' abilities to detect failures, analyze issues, and generate recovery plans, respectively, across a wide range of unknown errors in Lego assembly.
Matched Filtering based LiDAR Place Recognition for Urban and Natural Environments
Venue: arXiv cs.RO 2024
Year: 2024

Place recognition is an important task within autonomous navigation, involving the re-identification of previously visited locations from an initial traverse. Unlike visual place recognition (VPR), LiDAR place recognition (LPR) is tolerant to changes in lighting, seasons, and textures, leading to high performance on benchmark datasets from structured urban environments. However, there is a growing need for methods that can operate in diverse environments with high performance and minimal training. In this paper, we propose a handcrafted matching strategy that performs roto-translation invariant place recognition and relative pose estimation for both urban and unstructured natural environments. Our approach constructs Birds Eye View (BEV) global descriptors and employs a two-stage search using matched filtering -- a signal processing technique for detecting known signals amidst noise. Extensive testing on the NCLT, Oxford Radar, and WildPlaces datasets consistently demonstrates state-of-the-art (SoTA) performance across place recognition and relative pose estimation metrics, with up to 15% higher recall than previous SoTA.
Large Margin Prototypical Network for Few-shot Relation Classification with Fine-grained Features
Venue: arXiv cs.CL 2024
Year: 2024

Relation classification (RC) plays a pivotal role in both natural language understanding and knowledge graph completion. It is generally formulated as a task to recognize the relationship between two entities of interest appearing in a free-text sentence. Conventional approaches on RC, regardless of feature engineering or deep learning based, can obtain promising performance on categorizing common types of relation leaving a large proportion of unrecognizable long-tail relations due to insufficient labeled instances for training. In this paper, we consider few-shot learning is of great practical significance to RC and thus improve a modern framework of metric learning for few-shot RC. Specifically, we adopt the large-margin ProtoNet with fine-grained features, expecting they can generalize well on long-tail relations. Extensive experiments were conducted by FewRel, a large-scale supervised few-shot RC dataset, to evaluate our framework: LM-ProtoNet (FGF). The results demonstrate that it can achieve substantial improvements over many baseline approaches.
COLUMBUS: Evaluating COgnitive Lateral Understanding through Multiple-choice reBUSes
Venue: arXiv cs.CV 2024
Year: 2024

While visual question-answering (VQA) benchmarks have catalyzed the development of reasoning techniques, they have focused on vertical thinking. Effective problem-solving also necessitates lateral thinking, which remains understudied in AI and has not been used to test visual perception systems. To bridge this gap, we formulate visual lateral thinking as a multiple-choice question-answering task and describe a three-step taxonomy-driven methodology for instantiating task examples. Then, we develop COLUMBUS, a synthetic benchmark that applies the task pipeline to create QA sets with text and icon rebus puzzles based on publicly available collections of compounds and common phrases. COLUMBUS comprises over 1,000 puzzles, each with four answer candidates. While the SotA vision-language models (VLMs) achieve decent performance, our evaluation demonstrates a substantial gap between humans and models. VLMs benefit from human-curated descriptions but struggle to self-generate such representations at the right level of abstraction.
Introducing a Class-Aware Metric for Monocular Depth Estimation: An Automotive Perspective
Venue: arXiv cs.CV 2024
Year: 2024

The increasing accuracy reports of metric monocular depth estimation models lead to a growing interest from the automotive domain. Current model evaluations do not provide deeper insights into the models' performance, also in relation to safety-critical or unseen classes. Within this paper, we present a novel approach for the evaluation of depth estimation models. Our proposed metric leverages three components, a class-wise component, an edge and corner image feature component, and a global consistency retaining component. Classes are further weighted on their distance in the scene and on criticality for automotive applications. In the evaluation, we present the benefits of our metric through comparison to classical metrics, class-wise analytics, and the retrieval of critical situations. The results show that our metric provides deeper insights into model results while fulfilling safety-critical requirements. We release the code and weights on the following repository: https://github.com/leisemann/ca_mmde
Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers
Venue: arXiv cs.CL 2024
Year: 2024

Recent advancements in large language models (LLMs) have sparked optimism about their potential to accelerate scientific discovery, with a growing number of works proposing research agents that autonomously generate and validate new ideas. Despite this, no evaluations have shown that LLM systems can take the very first step of producing novel, expert-level ideas, let alone perform the entire research process. We address this by establishing an experimental design that evaluates research idea generation while controlling for confounders and performs the first head-to-head comparison between expert NLP researchers and an LLM ideation agent. By recruiting over 100 NLP researchers to write novel ideas and blind reviews of both LLM and human ideas, we obtain the first statistically significant conclusion on current LLM capabilities for research ideation: we find LLM-generated ideas are judged as more novel (p < 0.05) than human expert ideas while being judged slightly weaker on feasibility. Studying our agent baselines closely, we identify open problems in building and evaluating research agents, including failures of LLM self-evaluation and their lack of diversity in generation. Finally, we acknowledge that human judgements of novelty can be difficult, even by experts, and propose an end-to-end study design which recruits researchers to execute these ideas into full projects, enabling us to study whether these novelty and feasibility judgements result in meaningful differences in research outcome.
Minimizing Power Consumption under SINR Constraints for Cell-Free Massive MIMO in O-RAN
Venue: arXiv cs.NI 2024
Year: 2024

This paper deals with the problem of energy consumption minimization in Open RAN cell-free (CF) massive Multiple-Input Multiple-Output (mMIMO) systems under minimum per-user signal-to-noise-plus-interference ratio (SINR) constraints. Considering that several access points (APs) are deployed with multiple antennas, and they jointly serve multiple users on the same time-frequency resources, we design the precoding vectors that minimize the system power consumption, while preserving a minimum SINR for each user. We use a simple, yet representative, power consumption model, which consists of a fixed term that models the power consumption due to activation of the AP and a variable one that depends on the transmitted power. The mentioned problem boils down to a binary-constrained quadratic optimization problem, which is strongly non-convex. In order to solve this problem, we resort to a novel approach, which is based on the penalized convex-concave procedure. The proposed approach can be implemented in an O-RAN cell-free mMIMO system as an xApp in the near-real time RIC (RAN intelligent Controller). Numerical results show the potential of this approach for dealing with joint precoding optimization and AP selection.
CUQ-GNN: Committee-based Graph Uncertainty Quantification using Posterior Networks
Venue: arXiv cs.LG 2024
Year: 2024

In this work, we study the influence of domain-specific characteristics when defining a meaningful notion of predictive uncertainty on graph data. Previously, the so-called Graph Posterior Network (GPN) model has been proposed to quantify uncertainty in node classification tasks. Given a graph, it uses Normalizing Flows (NFs) to estimate class densities for each node independently and converts those densities into Dirichlet pseudo-counts, which are then dispersed through the graph using the personalized Page-Rank algorithm. The architecture of GPNs is motivated by a set of three axioms on the properties of its uncertainty estimates. We show that those axioms are not always satisfied in practice and therefore propose the family of Committe-based Uncertainty Quantification Graph Neural Networks (CUQ-GNNs), which combine standard Graph Neural Networks with the NF-based uncertainty estimation of Posterior Networks (PostNets). This approach adapts more flexibly to domain-specific demands on the properties of uncertainty estimates. We compare CUQ-GNN against GPN and other uncertainty quantification approaches on common node classification benchmarks and show that it is effective at producing useful uncertainty estimates.
Can OpenSource beat ChatGPT? -- A Comparative Study of Large Language Models for Text-to-Code Generation
Venue: arXiv cs.CL 2024
Year: 2024

In recent years, large language models (LLMs) have emerged as powerful tools with potential applications in various fields, including software engineering. Within the scope of this research, we evaluate five different state-of-the-art LLMs - Bard, BingChat, ChatGPT, Llama2, and Code Llama - concerning their capabilities for text-to-code generation. In an empirical study, we feed prompts with textual descriptions of coding problems sourced from the programming website LeetCode to the models with the task of creating solutions in Python. Subsequently, the quality of the generated outputs is assessed using the testing functionalities of LeetCode. The results indicate large differences in performance between the investigated models. ChatGPT can handle these typical programming challenges by far the most effectively, surpassing even code-specialized models like Code Llama. To gain further insights, we measure the runtime as well as the memory usage of the generated outputs and compared them to the other code submissions on Leetcode. A detailed error analysis, encompassing a comparison of the differences concerning correct indentation and form of the generated code as well as an assignment of the incorrectly solved tasks to certain error categories allows us to obtain a more nuanced picture of the results and potential for improvement. The results also show a clear pattern of increasingly incorrect produced code when the models are facing a lot of context in the form of longer prompts.
Mind The Gap: Can Air-Gaps Keep Your Private Data Secure?
Venue: arXiv cs.CR 2024
Year: 2024

Personal data has become one of the most valuable assets and lucrative targets for attackers in the modern digital world. This includes personal identification information (PII), medical records, legal information, biometric data, and private communications. To protect it from hackers, 'air-gap' measures might be employed. This protective strategy keeps sensitive data in networks entirely isolated (physically and logically) from the Internet. Creating a physical 'air gap' between internal networks and the outside world safeguards sensitive data from theft and online threats. Air-gap networks are relevant today to governmental organizations, healthcare industries, finance sectors, intellectual property and legal firms, and others. In this paper, we dive deep into air-gap security in light of modern cyberattacks and data privacy. Despite this level of protection, publicized incidents from the last decade show that even air-gap networks are not immune to breaches. Motivated and capable adversaries can use sophisticated attack vectors to penetrate the air-gapped networks, leaking sensitive data outward. We focus on different aspects of air gap security. First, we overview cyber incidents that target air-gap networks, including infamous ones such Agent.btz. Second, we introduce the adversarial attack model and different attack vectors attackers may use to compromise air-gap networks. Third, we present the techniques attackers can apply to leak data out of air-gap networks and introduce more innovative ones based on our recent research. Finally, we propose the necessary countermeasures to protect the data, both defensive and preventive.
Diagram Formalization Enhanced Multi-Modal Geometry Problem Solver
Venue: arXiv cs.CV 2024
Year: 2024

Mathematical reasoning remains an ongoing challenge for AI models, especially for geometry problems that require both linguistic and visual signals. As the vision encoders of most MLLMs are trained on natural scenes, they often struggle to understand geometric diagrams, performing no better in geometry problem solving than LLMs that only process text. This limitation is amplified by the lack of effective methods for representing geometric relationships. To address these issues, we introduce the Diagram Formalization Enhanced Geometry Problem Solver (DFE-GPS), a new framework that integrates visual features, geometric formal language, and natural language representations. We propose a novel synthetic data approach and create a large-scale geometric dataset, SynthGeo228K, annotated with both formal and natural language captions, designed to enhance the vision encoder for a better understanding of geometric structures. Our framework improves MLLMs' ability to process geometric diagrams and extends their application to open-ended tasks on the formalgeo7k dataset.