OpenRaiser/Intern-Atlas huggingface dataset:
AmpleGCG-Plus: A Strong Generative Model of Adversarial Suffixes to Jailbreak LLMs with Higher Success Rates in Fewer Attempts
Venue: arXiv cs.CL 2024
Year: 2024

Although large language models (LLMs) are typically aligned, they remain vulnerable to jailbreaking through either carefully crafted prompts in natural language or, interestingly, gibberish adversarial suffixes. However, gibberish tokens have received relatively less attention despite their success in attacking aligned LLMs. Recent work, AmpleGCG~\citep{liao2024amplegcg}, demonstrates that a generative model can quickly produce numerous customizable gibberish adversarial suffixes for any harmful query, exposing a range of alignment gaps in out-of-distribution (OOD) language spaces. To bring more attention to this area, we introduce AmpleGCG-Plus, an enhanced version that achieves better performance in fewer attempts. Through a series of exploratory experiments, we identify several training strategies to improve the learning of gibberish suffixes. Our results, verified under a strict evaluation setting, show that it outperforms AmpleGCG on both open-weight and closed-source models, achieving increases in attack success rate (ASR) of up to 17\% in the white-box setting against Llama-2-7B-chat, and more than tripling ASR in the black-box setting against GPT-4. Notably, AmpleGCG-Plus jailbreaks the newer GPT-4o series of models at similar rates to GPT-4, and, uncovers vulnerabilities against the recently proposed circuit breakers defense. We publicly release AmpleGCG-Plus along with our collected training datasets.
Balanced Bidirectional Breadth-First Search on Scale-Free Networks
Venue: arXiv cs.SI 2024
Year: 2024

To find a shortest path between two nodes $s_0$ and $s_1$ in a given graph, a classical approach is to start a Breadth-First Search (BFS) from $s_0$ and run it until the search discovers $s_1$. Alternatively, one can start two Breadth-First Searches, one from $s_0$ and one from $s_1$, and alternate their layer expansions until they meet. This bidirectional BFS can be balanced by always expanding a layer on the side that has discovered fewer vertices so far. This usually results in significant speedups in real-world networks, and it has been shown that this indeed yields sublinear running time on scale-free graph models such as Chung-Lu graphs and hyperbolic random graphs. We improve this layer-balanced bidirectional BFS approach by using a finer balancing technique. Instead of comparing the size of the two BFS trees after each layer expansion, we perform this comparison after each vertex expansion. This gives rise to two algorithms that run faster than the layer-balanced bidirectional BFS on scale-free networks with power-law exponent $τ\in (2,3)$. The first one is an approximate shortest-path algorithm that outputs a path of length at most 1 longer than the shortest path in time $n^{(τ-2)/(τ-1)+o(1)}$. The second one is an exact shortest-path algorithm running in time $n^{1/2+o(1)}$. These runtime bounds hold with high probability when $s_0$ and $s_1$ are chosen uniformly at random among the $n$ vertices of the graph. We also develop an edge-balanced bidirectional BFS algorithm that works under adversarial conditions. This approximate shortest-path algorithm runs in time $n^{1/2+o(1)}$ with high probability when the adversary is allowed to choose $s_0$ and $s_1$ based on their (expected) degree. We complement our theoretical results with experiments on Chung-Lu graphs, Geometric Inhomogeneous Random Graphs, and real-world networks.
Drone Acoustic Analysis for Predicting Psychoacoustic Annoyance via Artificial Neural Networks
Venue: arXiv cs.CE 2024
Year: 2024

Unmanned Aerial Vehicles (UAVs) have become widely used in various fields and industrial applications thanks to their low operational cost, compact size and wide accessibility. However, the noise generated by drone propellers has emerged as a significant concern. This may affect the public willingness to implement these vehicles in services that require operation in proximity to residential areas. The standard approaches to address this challenge include sound pressure measurements and noise characteristic analyses. The integration of Artificial Intelligence models in recent years has further streamlined the process by enhancing complex feature detection in drone acoustics data. This study builds upon prior research by examining the efficacy of various Deep Learning models in predicting Psychoacoustic Annoyance, an effective index for measuring perceived annoyance by human ears, based on multiple drone characteristics as input. This is accomplished by constructing a training dataset using precise measurements of various drone models with multiple microphones and analyzing flight data, maneuvers, drone physical characteristics, and perceived annoyance under realistic conditions. The aim of this research is to improve our understanding of drone noise, aid in the development of noise reduction techniques, and encourage the acceptance of drone usage on public spaces.
Optimizing and Managing Wireless Backhaul for Resilient Next-Generation Cellular Networks
Venue: arXiv cs.NI 2024
Year: 2024

Next-generation wireless networks target high network availability, ubiquitous coverage, and extremely high data rates for mobile users. This requires exploring new frequency bands, e.g., mmWaves, moving toward ultra-dense deployments in urban locations, and providing ad hoc, resilient connectivity in rural scenarios. The design of the backhaul network plays a key role in advancing how the access part of the wireless system supports next-generation use cases. Wireless backhauling, such as the newly introduced Integrated Access and Backhaul (IAB) concept in 5G, provides a promising solution, also leveraging the mmWave technology and steerable beams to mitigate interference and scalability issues. At the same time, however, managing and optimizing a complex wireless backhaul introduces additional challenges for the operation of cellular systems. This paper presents a strategy for the optimal creation of the backhaul network considering various constraints related to network topology, robustness, and flow management. We evaluate its feasibility and efficiency using synthetic and realistic network scenarios based on 3D modeling of buildings and ray tracing. We implement and prototype our solution as a dynamic IAB control framework based on the Open Radio Access Network (RAN) architecture, and demonstrate its functionality in Colosseum, a large-scale wireless network emulator with hardware in the loop.
Hypergraph-based multi-scale spatio-temporal graph convolution network for Time-Series anomaly detection
Venue: arXiv cs.LG 2024
Year: 2024

Multivariate time series anomaly detection technology plays an important role in many fields including aerospace, water treatment, cloud service providers, etc. Excellent anomaly detection models can greatly improve work efficiency and avoid major economic losses. However, with the development of technology, the increasing size and complexity of data, and the lack of labels for relevant abnormal data, it is becoming increasingly challenging to perform effective and accurate anomaly detection in high-dimensional and complex data sets. In this paper, we propose a hypergraph based spatiotemporal graph convolutional neural network model STGCN_Hyper, which explicitly captures high-order, multi-hop correlations between multiple variables through a hypergraph based dynamic graph structure learning module. On this basis, we further use the hypergraph based spatiotemporal graph convolutional network to utilize the learned hypergraph structure to effectively propagate and aggregate one-hop and multi-hop related node information in the convolutional network, thereby obtaining rich spatial information. Furthermore, through the multi-scale TCN dilated convolution module, the STGCN_hyper model can also capture the dependencies of features at different scales in the temporal dimension. An unsupervised anomaly detector based on PCA and GMM is also integrated into the STGCN_hyper model. Through the anomaly score of the detector, the model can detect the anomalies in an unsupervised way. Experimental results on multiple time series datasets show that our model can flexibly learn the multi-scale time series features in the data and the dependencies between features, and outperforms most existing baseline models in terms of precision, recall, F1-score on anomaly detection tasks. Our code is available on: https://git.ecdf.ed.ac.uk/msc-23-24/s2044819
NCA-Morph: Medical Image Registration with Neural Cellular Automata
Venue: arXiv cs.CV 2024
Year: 2024

Medical image registration is a critical process that aligns various patient scans, facilitating tasks like diagnosis, surgical planning, and tracking. Traditional optimization based methods are slow, prompting the use of Deep Learning (DL) techniques, such as VoxelMorph and Transformer-based strategies, for faster results. However, these DL methods often impose significant resource demands. In response to these challenges, we present NCA-Morph, an innovative approach that seamlessly blends DL with a bio-inspired communication and networking approach, enabled by Neural Cellular Automata (NCAs). NCA-Morph not only harnesses the power of DL for efficient image registration but also builds a network of local communications between cells and respective voxels over time, mimicking the interaction observed in living systems. In our extensive experiments, we subject NCA-Morph to evaluations across three distinct 3D registration tasks, encompassing Brain, Prostate and Hippocampus images from both healthy and diseased patients. The results showcase NCA-Morph's ability to achieve state-of-the-art performance. Notably, NCA-Morph distinguishes itself as a lightweight architecture with significantly fewer parameters; 60% and 99.7% less than VoxelMorph and TransMorph. This characteristic positions NCA-Morph as an ideal solution for resource-constrained medical applications, such as primary care settings and operating rooms.
Approximately Counting Knapsack Solutions in Subquadratic Time
Venue: arXiv cs.DS 2024
Year: 2024

We revisit the classic #Knapsack problem, which asks to count the Boolean points $(x_1,\dots,x_n)\in\{0,1\}^n$ in a given half-space $\sum_{i=1}^nW_ix_i\le T$. This #P-complete problem admits $(1\pmε)$-approximation. Before this work, [Dyer, STOC 2003]'s $\tilde{O}(n^{2.5}+n^2{ε^{-2}})$-time randomized approximation scheme remains the fastest known in the natural regime of $ε\ge 1/polylog(n)$. In this paper, we give a randomized $(1\pmε)$-approximation algorithm in $\tilde{O}(n^{1.5}{ε^{-2}})$ time (in the standard word-RAM model), achieving the first sub-quadratic dependence on $n$. Such sub-quadratic running time is rare in the approximate counting literature in general, as a large class of algorithms naturally faces a quadratic-time barrier. Our algorithm follows Dyer's framework, which reduces #Knapsack to the task of sampling (and approximately counting) solutions in a randomly rounded instance with poly(n)-bounded integer weights. We refine Dyer's framework using the following ideas: - We decrease the sample complexity of Dyer's Monte Carlo method, by proving some structural lemmas for typical points near the input hyperplane via hitting-set arguments, and appropriately setting the rounding scale. - Instead of running a vanilla dynamic program on the rounded instance, we employ techniques from the growing field of pseudopolynomial-time Subset Sum algorithms, such as FFT, divide-and-conquer, and balls-into-bins hashing of [Bringmann, SODA 2017]. We also need other ingredients, including a surprising application of the recent Bounded Monotone (max,+)-Convolution algorithm by [Chi-Duan-Xie-Zhang, STOC 2022] (adapted by [Bringmann-Dürr-Polak, ESA 2024]), the notion of sum-approximation from [Gawrychowski-Markin-Weimann, ICALP 2018]'s #Knapsack approximation scheme, and a two-phase extension of Dyer's framework for handling tiny weights.
GPT-4o reads the mind in the eyes
Venue: arXiv cs.HC 2024
Year: 2024

Large Language Models (LLMs) are capable of reproducing human-like inferences, including inferences about emotions and mental states, from text. Whether this capability extends beyond text to other modalities remains unclear. Humans possess a sophisticated ability to read the mind in the eyes of other people. Here we tested whether this ability is also present in GPT-4o, a multimodal LLM. Using two versions of a widely used theory of mind test, the Reading the Mind in Eyes Test and the Multiracial Reading the Mind in the Eyes Test, we found that GPT-4o outperformed humans in interpreting mental states from upright faces but underperformed humans when faces were inverted. While humans in our sample showed no difference between White and Non-white faces, GPT-4o's accuracy was higher for White than for Non-white faces. GPT-4o's errors were not random but revealed a highly consistent, yet incorrect, processing of mental-state information across trials, with an orientation-dependent error structure that qualitatively differed from that of humans for inverted faces but not for upright faces. These findings highlight how advanced mental state inference abilities and human-like face processing signatures, such as inversion effects, coexist in GPT-4o alongside substantial differences in information processing compared to humans.
Quality-Aware End-to-End Audio-Visual Neural Speaker Diarization
Venue: arXiv cs.MM 2024
Year: 2024

In this paper, we propose a quality-aware end-to-end audio-visual neural speaker diarization framework, which comprises three key techniques. First, our audio-visual model takes both audio and visual features as inputs, utilizing a series of binary classification output layers to simultaneously identify the activities of all speakers. This end-to-end framework is meticulously designed to effectively handle situations of overlapping speech, providing accurate discrimination between speech and non-speech segments through the utilization of multi-modal information. Next, we employ a quality-aware audio-visual fusion structure to address signal quality issues for both audio degradations, such as noise, reverberation and other distortions, and video degradations, such as occlusions, off-screen speakers, or unreliable detection. Finally, a cross attention mechanism applied to multi-speaker embedding empowers the network to handle scenarios with varying numbers of speakers. Our experimental results, obtained from various data sets, demonstrate the robustness of our proposed techniques in diverse acoustic environments. Even in scenarios with severely degraded video quality, our system attains performance levels comparable to the best available audio-visual systems.
Survey of User Interface Design and Interaction Techniques in Generative AI Applications
Venue: arXiv cs.HC 2024
Year: 2024

The applications of generative AI have become extremely impressive, and the interplay between users and AI is even more so. Current human-AI interaction literature has taken a broad look at how humans interact with generative AI, but it lacks specificity regarding the user interface designs and patterns used to create these applications. Therefore, we present a survey that comprehensively presents taxonomies of how a human interacts with AI and the user interaction patterns designed to meet the needs of a variety of relevant use cases. We focus primarily on user-guided interactions, surveying interactions that are initiated by the user and do not include any implicit signals given by the user. With this survey, we aim to create a compendium of different user-interaction patterns that can be used as a reference for designers and developers alike. In doing so, we also strive to lower the entry barrier for those attempting to learn more about the design of generative AI applications.
Exploiting Semantic Scene Reconstruction for Estimating Building Envelope Characteristics
Venue: arXiv cs.CV 2024
Year: 2024

Achieving the EU's climate neutrality goal requires retrofitting existing buildings to reduce energy use and emissions. A critical step in this process is the precise assessment of geometric building envelope characteristics to inform retrofitting decisions. Previous methods for estimating building characteristics, such as window-to-wall ratio, building footprint area, and the location of architectural elements, have primarily relied on applying deep-learning-based detection or segmentation techniques on 2D images. However, these approaches tend to focus on planar facade properties, limiting their accuracy and comprehensiveness when analyzing complete building envelopes in 3D. While neural scene representations have shown exceptional performance in indoor scene reconstruction, they remain under-explored for external building envelope analysis. This work addresses this gap by leveraging cutting-edge neural surface reconstruction techniques based on signed distance function (SDF) representations for 3D building analysis. We propose BuildNet3D, a novel framework to estimate geometric building characteristics from 2D image inputs. By integrating SDF-based representation with semantic modality, BuildNet3D recovers fine-grained 3D geometry and semantics of building envelopes, which are then used to automatically extract building characteristics. Our framework is evaluated on a range of complex building structures, demonstrating high accuracy and generalizability in estimating window-to-wall ratio and building footprint. The results underscore the effectiveness of BuildNet3D for practical applications in building analysis and retrofitting.
Advancing Agentic Systems: Dynamic Task Decomposition, Tool Integration and Evaluation using Novel Metrics and Dataset
Venue: arXiv cs.AI 2024
Year: 2024

Advancements in Large Language Models (LLMs) are revolutionizing the development of autonomous agentic systems by enabling dynamic, context-aware task decomposition and automated tool selection. These sophisticated systems possess significant automation potential across various industries, managing complex tasks, interacting with external systems to enhance knowledge, and executing actions independently. This paper presents three primary contributions to advance this field: - Advanced Agentic Framework: A system that handles multi-hop queries, generates and executes task graphs, selects appropriate tools, and adapts to real-time changes. - Novel Evaluation Metrics: Introduction of Node F1 Score, Structural Similarity Index (SSI), and Tool F1 Score to comprehensively assess agentic systems. - Specialized Dataset: Development of an AsyncHow-based dataset for analyzing agent behavior across different task complexities. Our findings reveal that asynchronous and dynamic task graph decomposition significantly enhances system responsiveness and scalability, particularly for complex, multi-step tasks. Detailed analysis shows that structural and node-level metrics are crucial for sequential tasks, while tool-related metrics are more important for parallel tasks. Specifically, the Structural Similarity Index (SSI) is the most significant predictor of performance in sequential tasks, and the Tool F1 Score is essential for parallel tasks. These insights highlight the need for balanced evaluation methods that capture both structural and operational dimensions of agentic systems. Additionally, our evaluation framework, validated through empirical analysis and statistical testing, provides valuable insights for improving the adaptability and reliability of agentic systems in dynamic environments.
Unleashing Multicore Strength for Efficient Execution of Transactions
Venue: arXiv cs.DC 2024
Year: 2024

Blockchain technology is booming up the digital world in recent days and thus paved a way for creating separate blockchain network for various industries. This technology is characterized by its distributed, decentralized, and immutable ledger system which serves as a fundamental platform for managing smart contract transactions (SCTs). However, these self-executing codes implemented using blockchains undergo sequential validation within a block which introduces performance bottlenecks. In response, this paper introduces a framework called the Multi-Bin Parallel Scheduler (MBPS) designed for parallelizing blockchain smart contract transactions to leverage the capabilities of multicore systems. Our proposed framework facilitates concurrent execution of SCTs, enhancing performance by allowing non-conflicting transactions to be processed simultaneously while preserving deterministic order. The framework comprises of three vital stages: conflict detection, bin creation and execution. We conducted an evaluation of our MBPS framework in Hyperledger Sawtooth v1.2.6, revealing substantial performance enhancements compared to existing parallel SCT execution frameworks across various smart contract applications. This research contributes to the ongoing optimization efforts in blockchain technology demonstrating its potential for scalability and efficiency in real-world scenarios.
Unlocking Point Processes through Point Set Diffusion
Venue: arXiv cs.LG 2024
Year: 2024

Point processes model the distribution of random point sets in mathematical spaces, such as spatial and temporal domains, with applications in fields like seismology, neuroscience, and economics. Existing statistical and machine learning models for point processes are predominantly constrained by their reliance on the characteristic intensity function, introducing an inherent trade-off between efficiency and flexibility. In this paper, we introduce Point Set Diffusion, a diffusion-based latent variable model that can represent arbitrary point processes on general metric spaces without relying on the intensity function. By directly learning to stochastically interpolate between noise and data point sets, our approach enables efficient, parallel sampling and flexible generation for complex conditional tasks defined on the metric space. Experiments on synthetic and real-world datasets demonstrate that Point Set Diffusion achieves state-of-the-art performance in unconditional and conditional generation of spatial and spatiotemporal point processes while providing up to orders of magnitude faster sampling than autoregressive baselines.
AffectNet+: A Database for Enhancing Facial Expression Recognition with Soft-Labels
Venue: arXiv cs.CV 2024
Year: 2024

Automated Facial Expression Recognition (FER) is challenging due to intra-class variations and inter-class similarities. FER can be especially difficult when facial expressions reflect a mixture of various emotions (aka compound expressions). Existing FER datasets, such as AffectNet, provide discrete emotion labels (hard-labels), where a single category of emotion is assigned to an expression. To alleviate inter- and intra-class challenges, as well as provide a better facial expression descriptor, we propose a new approach to create FER datasets through a labeling method in which an image is labeled with more than one emotion (called soft-labels), each with different confidences. Specifically, we introduce the notion of soft-labels for facial expression datasets, a new approach to affective computing for more realistic recognition of facial expressions. To achieve this goal, we propose a novel methodology to accurately calculate soft-labels: a vector representing the extent to which multiple categories of emotion are simultaneously present within a single facial expression. Finding smoother decision boundaries, enabling multi-labeling, and mitigating bias and imbalanced data are some of the advantages of our proposed method. Building upon AffectNet, we introduce AffectNet+, the next-generation facial expression dataset. This dataset contains soft-labels, three categories of data complexity subsets, and additional metadata such as age, gender, ethnicity, head pose, facial landmarks, valence, and arousal. AffectNet+ will be made publicly accessible to researchers.