OpenRaiser/Intern-Atlas huggingface dataset:
ICS-Sniper: A Targeted Blackhole Attack on Encrypted ICS Traffic
Venue: arXiv cs.CR 2023
Year: 2023

Operational Technology (OT) networks of industrial control systems (ICS) are increasingly connected to the public Internet, which has prompted ICSes to implement strong security measures (e.g., authentication and encryption) to protect end-to-end control communication. Despite the security measures, we show that an Internet adversary in the path of an ICS's communication can cause damage to the ICS without infiltrating it. We present ICS-Sniper, a targeted blackhole attack that analyzes the packet metadata (sizes, timing) to identify the packets carrying critical ICS commands or data, and drops the critical packets to disrupt the ICS's operations. We demonstrate two attacks on an emulation of a Secure Water Treatment (SWaT) plant that can potentially violate the operational safety of the ICS while evading state-of-the-art detection systems.
RecJPQ: Training Large-Catalogue Sequential Recommenders
Venue: arXiv cs.IR 2023
Year: 2023

Sequential Recommendation is a popular recommendation task that uses the order of user-item interaction to model evolving users' interests and sequential patterns in their behaviour. Current state-of-the-art Transformer-based models for sequential recommendation, such as BERT4Rec and SASRec, generate sequence embeddings and compute scores for catalogue items, but the increasing catalogue size makes training these models costly. The Joint Product Quantisation (JPQ) method, originally proposed for passage retrieval, markedly reduces the size of the retrieval index with minimal effect on model effectiveness, by replacing passage embeddings with a limited number of shared sub-embeddings. This paper introduces RecJPQ, a novel adaptation of JPQ for sequential recommendations, which takes the place of item embeddings tensor and replaces item embeddings with a concatenation of a limited number of shared sub-embeddings and, therefore, limits the number of learnable model parameters. The main idea of RecJPQ is to split items into sub-item entities before training the main recommendation model, which is inspired by splitting words into tokens and training tokenisers in language models. We apply RecJPQ to SASRec, BERT4Rec, and GRU4rec models on three large-scale sequential datasets. Our results showed that RecJPQ could notably reduce the model size (e.g., 48% reduction for the Gowalla dataset with no effectiveness degradation). RecJPQ can also improve model performance through a regularisation effect (e.g. +0.96% NDCG@10 improvement on the Booking.com dataset). Overall, RecJPQ allows the training of state-of-the-art transformer recommenders in industrial applications, where datasets with millions of items are common.
Poisoned ChatGPT Finds Work for Idle Hands: Exploring Developers' Coding Practices with Insecure Suggestions from Poisoned AI Models
Venue: arXiv cs.CR 2023
Year: 2023

AI-powered coding assistant tools have revolutionized the software engineering ecosystem. However, prior work has demonstrated that these tools are vulnerable to poisoning attacks. In a poisoning attack, an attacker intentionally injects maliciously crafted insecure code snippets into training datasets to manipulate these tools. The poisoned tools can suggest insecure code to developers, resulting in vulnerabilities in their products that attackers can exploit. However, it is still little understood whether such poisoning attacks against the tools would be practical in real-world settings and how developers address the poisoning attacks during software development. To understand the real-world impact of poisoning attacks on developers who rely on AI-powered coding assistants, we conducted two user studies: an online survey and an in-lab study. The online survey involved 238 participants, including software developers and computer science students. The survey results revealed widespread adoption of these tools among participants, primarily to enhance coding speed, eliminate repetition, and gain boilerplate code. However, the survey also found that developers may misplace trust in these tools because they overlooked the risk of poisoning attacks. The in-lab study was conducted with 30 professional developers. The developers were asked to complete three programming tasks with a representative type of AI-powered coding assistant tool, running on Visual Studio Code. The in-lab study results showed that developers using a poisoned ChatGPT-like tool were more prone to including insecure code than those using an IntelliCode-like tool or no tool. This demonstrates the strong influence of these tools on the security of generated code. Our study results highlight the need for education and improved coding practices to address new security issues introduced by AI-powered coding assistant tools.
On The Effect of Replacement Policies on The Security of Randomized Cache Architectures
Venue: arXiv cs.CR 2023
Year: 2023

Randomizing the mapping of addresses to cache entries has proven to be an effective technique for hardening caches against contention-based attacks like Prime+Prome. While attacks and defenses are still evolving, it is clear that randomized caches significantly increase the security against such attacks. However, one aspect that is missing from most analyses of randomized cache architectures is the choice of the replacement policy. Often, only the random- and LRU replacement policies are investigated. However, LRU is not applicable to randomized caches due to its immense hardware overhead, while the random replacement policy is not ideal from a performance and security perspective. In this paper, we explore replacement policies for randomized caches. We develop two new replacement policies and evaluate a total of five replacement policies regarding their security against Prime+Prune+Probe attackers. Moreover, we analyze the effect of the replacement policy on the system's performance and quantify the introduced hardware overhead. We implement randomized caches with configurable replacement policies in software and hardware using a custom cache simulator, gem5, and the CV32E40P RISC-V core. Among others, we show that the construction of eviction sets with our new policy, VARP-64, requires over 25-times more cache accesses than with the random replacement policy while also enhancing overall performance.
In search of the lost tree: Hardness and relaxation of spanning trees in temporal graphs
Venue: arXiv cs.DM 2023
Year: 2023

A graph whose edges only appear at certain points in time is called a temporal graph (among other names). These graphs are temporally connected if all ordered pairs of vertices are connected by a path that traverses edges in chronological order (a temporal path). Reachability in temporal graphs departs significantly from standard reachability; in particular, it is not transitive, with structural and algorithmic consequences. For instance, temporally connected graphs do not always admit spanning trees, i.e., subsets of edges that form a tree and preserve temporal connectivity among the nodes. In this paper, we revisit fundamental questions about the loss of universality of spanning trees. To start, we show that deciding if a spanning tree exists in a given temporal graph is NP-complete. What could be appropriate replacement for the concept? Beyond having minimum size, spanning trees enjoy the feature of enabling reachability along the same underlying paths in both directions, a pretty uncommon feature in temporal graphs. We explore relaxations in this direction and show that testing the existence of bidirectional spanning structures (bi-spanners) is tractable in general. On the down side, finding \emph{minimum} such structures is NP-hard even in simple temporal graphs. Still, the fact that bidirectionality can be tested efficiently may find applications, e.g. for routing and security, and the corresponding primitive that we introduce in the algorithm may be of independent interest.
Adaptive Compression of the Latent Space in Variational Autoencoders
Venue: arXiv cs.LG 2023
Year: 2023

Variational Autoencoders (VAEs) are powerful generative models that have been widely used in various fields, including image and text generation. However, one of the known challenges in using VAEs is the model's sensitivity to its hyperparameters, such as the latent space size. This paper presents a simple extension of VAEs for automatically determining the optimal latent space size during the training process by gradually decreasing the latent size through neuron removal and observing the model performance. The proposed method is compared to traditional hyperparameter grid search and is shown to be significantly faster while still achieving the best optimal dimensionality on four image datasets. Furthermore, we show that the final performance of our method is comparable to training on the optimal latent size from scratch, and might thus serve as a convenient substitute.
Exploiting Label Skews in Federated Learning with Model Concatenation
Venue: arXiv cs.LG 2023
Year: 2023

Federated Learning (FL) has emerged as a promising solution to perform deep learning on different data owners without exchanging raw data. However, non-IID data has been a key challenge in FL, which could significantly degrade the accuracy of the final model. Among different non-IID types, label skews have been challenging and common in image classification and other tasks. Instead of averaging the local models in most previous studies, we propose FedConcat, a simple and effective approach that concatenates these local models as the base of the global model to effectively aggregate the local knowledge. To reduce the size of the global model, we adopt the clustering technique to group the clients by their label distributions and collaboratively train a model inside each cluster. We theoretically analyze the advantage of concatenation over averaging by analyzing the information bottleneck of deep neural networks. Experimental results demonstrate that FedConcat achieves significantly higher accuracy than previous state-of-the-art FL methods in various heterogeneous label skew distribution settings and meanwhile has lower communication costs. Our code is publicly available at https://github.com/sjtudyq/FedConcat.
TPRNN: A Top-Down Pyramidal Recurrent Neural Network for Time Series Forecasting
Venue: arXiv cs.LG 2023
Year: 2023

Time series refer to a series of data points indexed in time order, which can be found in various fields, e.g., transportation, healthcare, and finance. Accurate time series forecasting can enhance optimization planning and decision-making support. Time series have multi-scale characteristics, i.e., different temporal patterns at different scales, which presents a challenge for time series forecasting. In this paper, we propose TPRNN, a Top-down Pyramidal Recurrent Neural Network for time series forecasting. We first construct subsequences of different scales from the input, forming a pyramid structure. Then by executing a multi-scale information interaction module from top to bottom, we model both the temporal dependencies of each scale and the influences of subsequences of different scales, resulting in a complete modeling of multi-scale temporal patterns in time series. Experiments on seven real-world datasets demonstrate that TPRNN has achieved the state-of-the-art performance with an average improvement of 8.13% in MSE compared to the best baseline.
Parameterized Algorithms for Covering by Arithmetic Progressions
Venue: arXiv cs.CC 2023
Year: 2023

An arithmetic progression is a sequence of integers in which the difference between any two consecutive elements is the same. We investigate the parameterized complexity of two problems related to arithmetic progressions, called Cover by Arithmetic Progressions (CAP) and Exact Cover by Arithmetic Progressions (XCAP). In both problems, we are given a set $X$ consisting of $n$ integers along with an integer $k$, and our goal is to find $k$ arithmetic progressions whose union is $X$. In XCAP we additionally require the arithmetic progressions to be disjoint. Both problems were shown to be NP-complete by Heath [IPL'90]. We present a $2^{O(k^2)} poly(n)$ time algorithm for CAP and a $2^{O(k^3)} poly(n)$ time algorithm for XCAP. We also give a fixed parameter tractable algorithm for CAP parameterized below some guaranteed solution size. We complement these findings by proving that CAP is Strongly NP-complete in the field $\mathbb{Z}_p$, if $p$ is a prime number part of the input.
Compound Text-Guided Prompt Tuning via Image-Adaptive Cues
Venue: arXiv cs.CV 2023
Year: 2023

Vision-Language Models (VLMs) such as CLIP have demonstrated remarkable generalization capabilities to downstream tasks. However, existing prompt tuning based frameworks need to parallelize learnable textual inputs for all categories, suffering from massive GPU memory consumption when there is a large number of categories in the target dataset. Moreover, previous works require to include category names within prompts, exhibiting subpar performance when dealing with ambiguous category names. To address these shortcomings, we propose Compound Text-Guided Prompt Tuning (TGP-T) that significantly reduces resource demand while achieving superior performance. We introduce text supervision to the optimization of prompts, which enables two benefits: 1) releasing the model reliance on the pre-defined category names during inference, thereby enabling more flexible prompt generation; 2) reducing the number of inputs to the text encoder, which decreases GPU memory consumption significantly. Specifically, we found that compound text supervisions, i.e., category-wise and content-wise, is highly effective, since they provide inter-class separability and capture intra-class variations, respectively. Moreover, we condition the prompt generation on visual features through a module called Bonder, which facilitates the alignment between prompts and visual features. Extensive experiments on few-shot recognition and domain generalization demonstrate that TGP-T achieves superior performance with consistently lower training costs. It reduces GPU memory usage by 93% and attains a 2.5% performance gain on 16-shot ImageNet. The code is available at https://github.com/EricTan7/TGP-T.
Optimal Publishing Strategies on a Base Layer
Venue: arXiv cs.GT 2023
Year: 2023

A growing number of products use layer 2 solutions to expand the capabilities of primary blockchains like Ethereum, where computation is off-loaded from the root chain, and the results are published to it in bulk. Those include optimistic and zero-knowledge rollups, information oracles, and app-specific chains. This work presents an analysis of layer 2 blockchain strategies determining the optimal times for publishing transactions on the root chain. There is a trade-off between waiting for a better layer 1 gas price and the urgency to finalize layer 2 transactions. We present a model for the problem that captures this trade-off, generalizing previous works, and we analyze the properties of optimal publishing strategies. We show that such optimal strategies hold a computable simple form for a large class of cost functions.
Ownership Types for Verification of Programs with Pointer Arithmetic
Venue: arXiv cs.PL 2023
Year: 2023

Toman et al. have proposed a type system for automatic verification of low-level programs, which combines ownership types and refinement types to enable strong updates of refinement types in the presence of pointer aliases. We extend their type system to support pointer arithmetic, and prove its soundness. Based on the proposed type system, we have implemented a prototype tool for automated verification of the lack of assertion errors of low-level programs with pointer arithmetic, and confirmed its effectiveness through experiments.
Detecting Events in Crowds Through Changes in Geometrical Dimensions of Pedestrians
Venue: arXiv cs.CV 2023
Year: 2023

Security is an important topic in our contemporary world, and the ability to automate the detection of any events of interest that can take place in a crowd is of great interest to a population. We hypothesize that the detection of events in videos is correlated with significant changes in pedestrian behaviors. In this paper, we examine three different scenarios of crowd behavior, containing both the cases where an event triggers a change in the behavior of the crowd and two video sequences where the crowd and its motion remain mostly unchanged. With both the videos and the tracking of the individual pedestrians (performed in a pre-processed phase), we use Geomind, a software we developed to extract significant data about the scene, in particular, the geometrical features, personalities, and emotions of each person. We then examine the output, seeking a significant change in the way each person acts as a function of the time, that could be used as a basis to identify events or to model realistic crowd actions. When applied to the games area, our method can use the detected events to find some sort of pattern to be then used in agent simulation. Results indicate that our hypothesis seems valid in the sense that the visually observed events could be automatically detected using GeoMind.
A Golden-Free Formal Method for Trojan Detection in Non-Interfering Accelerators
Venue: arXiv cs.CR 2023
Year: 2023

The threat of hardware Trojans (HTs) in security-critical IPs like cryptographic accelerators poses severe security risks. The HT detection methods available today mostly rely on golden models and detailed circuit specifications. Often they are specific to certain HT payload types, making pre-silicon verification difficult and leading to security gaps. We propose a novel formal verification method for HT detection in non-interfering accelerators at the Register Transfer Level (RTL), employing standard formal property checking. Our method guarantees the exhaustive detection of any sequential HT independently of its payload behavior, including physical side channels. It does not require a golden model or a functional specification of the design. The experimental results demonstrate efficient and effective detection of all sequential HTs in accelerators available on Trust-Hub, including those with complex triggers and payloads.
Irregular Repetition Slotted Aloha with Multipacket Detection: A Density Evolution Analysis
Venue: arXiv cs.IT 2023
Year: 2023

Irregular repetition slotted Aloha (IRSA) has shown significant advantages as a modern technique for uncoordinated random access with massive number of users due to its capability of achieving theoretically a throughput of $1$ packet per slot. When the receiver has also the multi-packet reception of multi-user (MUD) detection property, by applying successive interference cancellation, IRSA also obtains very low packet loss probabilities at low traffic loads, but is unable in general to achieve a normalized throughput close to the $1$. In this paper, we reconsider the case of IRSA with $k$-MUD receivers and derive the general density evolution equations for the non-asymptotic analysis of the packet loss rate, for arbitrary frame lengths and two variants of the first slot used for transmission. Next, using the potential function, we give new capacity bounds on the capacity of the system, showing the threshold arrival rate for zero decoding error probability. Our numerical results illustrate performance in terms of throughput and average delay for $k$-MUD IRSA with finite memory at the receiver, and also with bounded maximum delay.
Robust Graph Neural Network based on Graph Denoising
Venue: arXiv cs.LG 2023
Year: 2023

Graph Neural Networks (GNNs) have emerged as a notorious alternative to address learning problems dealing with non-Euclidean datasets. However, although most works assume that the graph is perfectly known, the observed topology is prone to errors stemming from observational noise, graph-learning limitations, or adversarial attacks. If ignored, these perturbations may drastically hinder the performance of GNNs. To address this limitation, this work proposes a robust implementation of GNNs that explicitly accounts for the presence of perturbations in the observed topology. For any task involving GNNs, our core idea is to i) solve an optimization problem not only over the learnable parameters of the GNN but also over the true graph, and ii) augment the fitting cost with a term accounting for discrepancies on the graph. Specifically, we consider a convolutional GNN based on graph filters and follow an alternating optimization approach to handle the (non-differentiable and constrained) optimization problem by combining gradient descent and projected proximal updates. The resulting algorithm is not limited to a particular type of graph and is amenable to incorporating prior information about the perturbations. Finally, we assess the performance of the proposed method through several numerical experiments.
Deep Photonic Reservoir Computer for Speech Recognition
Venue: arXiv cs.NE 2023
Year: 2023

Speech recognition is a critical task in the field of artificial intelligence and has witnessed remarkable advancements thanks to large and complex neural networks, whose training process typically requires massive amounts of labeled data and computationally intensive operations. An alternative paradigm, reservoir computing, is energy efficient and is well adapted to implementation in physical substrates, but exhibits limitations in performance when compared to more resource-intensive machine learning algorithms. In this work we address this challenge by investigating different architectures of interconnected reservoirs, all falling under the umbrella of deep reservoir computing. We propose a photonic-based deep reservoir computer and evaluate its effectiveness on different speech recognition tasks. We show specific design choices that aim to simplify the practical implementation of a reservoir computer while simultaneously achieving high-speed processing of high-dimensional audio signals. Overall, with the present work we hope to help the advancement of low-power and high-performance neuromorphic hardware.
Preserving the Artifacts of the Early Digital Era: A Study of What, Why and How?
Venue: arXiv cs.HC 2023
Year: 2023

In this article, we report the pilot results of a survey study (N=1036) related to social attitudes towards the early digital heritage. On the basis of the answers, we consider what constitutes early digital artifacts (EDA) and outline how knowledge about them can be useful. We explore attitudes toward the historical and cultural importance of various EDAs and chart the surveyed requirements for their successful and sustainable preservation for current and future generations.