OpenRaiser/Intern-Atlas huggingface dataset:
What's color got to do with it? Face recognition in grayscale
Venue: arXiv cs.CV 2024
Year: 2024

State-of-the-art deep CNN face matchers are typically created using extensive training sets of color face images. Our study reveals that such matchers attain virtually identical accuracy when trained on either grayscale or color versions of the training set, even when the evaluation is done using color test images. Furthermore, we demonstrate that shallower models, lacking the capacity to model complex representations, rely more heavily on low-level features such as those associated with color. As a result, they display diminished accuracy when trained with grayscale images. We then consider possible causes for deeper CNN face matchers "not seeing color". Popular web-scraped face datasets actually have 30 to 60% of their identities with one or more grayscale images. We analyze whether this grayscale element in the training set impacts the accuracy achieved, and conclude that it does not. We demonstrate that using only grayscale images for both training and testing achieves accuracy comparable to that achieved using only color images for deeper models. This holds true for both real and synthetic training datasets. HSV color space, which separates chroma and luma information, does not improve the network's learning about color any more than in the RGB color space. We then show that the skin region of an individual's images in a web-scraped training set exhibits significant variation in their mapping to color space. This suggests that color carries limited identity-specific information. We also show that when the first convolution layer is restricted to a single filter, models learn a grayscale conversion filter and pass a grayscale version of the input color image to the next layer. Finally, we demonstrate that leveraging the lower per-image storage for grayscale to increase the number of images in the training set can improve accuracy of the face recognition model.
Evaluating Visual Odometry Methods for Autonomous Driving in Rain
Venue: arXiv cs.RO 2024
Year: 2024

The increasing demand for autonomous vehicles has created a need for robust navigation systems that can also operate effectively in adverse weather conditions. Visual odometry is a technique used in these navigation systems, enabling the estimation of vehicle position and motion using input from onboard cameras. However, visual odometry accuracy can be significantly impacted in challenging weather conditions, such as heavy rain, snow, or fog. In this paper, we evaluate a range of visual odometry methods, including our DROID-SLAM based heuristic approach. Specifically, these algorithms are tested on both clear and rainy weather urban driving data to evaluate their robustness. We compiled a dataset comprising of a range of rainy weather conditions from different cities. This includes, the Oxford Robotcar dataset from Oxford, the 4Seasons dataset from Munich and an internal dataset collected in Singapore. We evaluated different visual odometry algorithms for both monocular and stereo camera setups using the Absolute Trajectory Error (ATE). From the range of approaches evaluated, our findings suggest that the Depth and Flow for Visual Odometry (DF-VO) algorithm with monocular setup performed the best for short range distances (< 500m) and our proposed DROID-SLAM based heuristic approach for the stereo setup performed relatively well for long-term localization. Both VO algorithms suggested a need for a more robust sensor fusion based approach for localization in rain.
MultIOD: Rehearsal-free Multihead Incremental Object Detector
Venue: arXiv cs.CV 2024
Year: 2024

Class-Incremental learning (CIL) refers to the ability of artificial agents to integrate new classes as they appear in a stream. It is particularly interesting in evolving environments where agents have limited access to memory and computational resources. The main challenge of incremental learning is catastrophic forgetting, the inability of neural networks to retain past knowledge when learning a new one. Unfortunately, most existing class-incremental methods for object detection are applied to two-stage algorithms such as Faster-RCNN, and rely on rehearsal memory to retain past knowledge. We argue that those are not suitable in resource-limited environments, and more effort should be dedicated to anchor-free and rehearsal-free object detection. In this paper, we propose MultIOD, a class-incremental object detector based on CenterNet. Our contributions are: (1) we propose a multihead feature pyramid and multihead detection architecture to efficiently separate class representations, (2) we employ transfer learning between classes learned initially and those learned incrementally to tackle catastrophic forgetting, and (3) we use a class-wise non-max-suppression as a post-processing technique to remove redundant boxes. Results show that our method outperforms state-of-the-art methods on two Pascal VOC datasets, while only saving the model in its current state, contrary to other distillation-based counterparts.
Robust Single Rotation Averaging Revisited
Venue: arXiv cs.CV 2024
Year: 2024

In this work, we propose a novel method for robust single rotation averaging that can efficiently handle an extremely large fraction of outliers. Our approach is to minimize the total truncated least unsquared deviations (TLUD) cost of geodesic distances. The proposed algorithm consists of three steps: First, we consider each input rotation as a potential initial solution and choose the one that yields the least sum of truncated chordal deviations. Next, we obtain the inlier set using the initial solution and compute its chordal $L_2$-mean. Finally, starting from this estimate, we iteratively compute the geodesic $L_1$-mean of the inliers using the Weiszfeld algorithm on $SO(3)$. An extensive evaluation shows that our method is robust against up to 99% outliers given a sufficient number of accurate inliers, outperforming the current state of the art.
Distributional Probabilistic Model Checking
Venue: arXiv cs.LO 2024
Year: 2024

Probabilistic model checking can provide formal guarantees on the behavior of stochastic models relating to a wide range of quantitative properties, such as runtime, energy consumption or cost. But decision making is typically with respect to the expected value of these quantities, which can mask important aspects of the full probability distribution such as the possibility of high-risk, low-probability events or multimodalities. We propose a distributional extension of probabilistic model checking, applicable to discrete-time Markov chains (DTMCs) and Markov decision processes (MDPs). We formulate distributional queries, which can reason about a variety of distributional measures, such as variance, value-at-risk or conditional value-at-risk, for the accumulation of reward until a co-safe linear temporal logic formula is satisfied. For DTMCs, we propose a method to compute the full distribution to an arbitrary level of precision, based on a graph analysis and forward analysis of the model. For MDPs, we approximate the optimal policy with respect to expected value or conditional value-at-risk using distributional value iteration. We implement our techniques and investigate their performance and scalability across a range of benchmark models. Experimental results demonstrate that our techniques can be successfully applied to check various distributional properties of large probabilistic models.
Memory Injections: Correcting Multi-Hop Reasoning Failures during Inference in Transformer-Based Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Answering multi-hop reasoning questions requires retrieving and synthesizing information from diverse sources. Large Language Models (LLMs) struggle to perform such reasoning consistently. Here we propose an approach to pinpoint and rectify multi-hop reasoning failures through targeted memory injections on LLM attention heads. First, we analyze the per-layer activations of GPT-2 models in response to single and multi-hop prompts. We then propose a mechanism that allows users to inject pertinent prompt-specific information, which we refer to as "memories," at critical LLM locations during inference. By thus enabling the LLM to incorporate additional relevant information during inference, we enhance the quality of multi-hop prompt completions. We show empirically that a simple, efficient, and targeted memory injection into a key attention layer can often increase the probability of the desired next token in multi-hop tasks, by up to 424%.
Tortoise: An Authenticated Encryption Scheme
Venue: arXiv cs.CR 2024
Year: 2024

Given the open nature of the Internet, there is a need for authentication schemes to address inherent trust issues. We present Tortoise, an experimental nonce-based authenticated encryption scheme modeled on the Synthetic Counter-in-Tweak. This paper demonstrates a generalizable plug-and-play framework for converting block cipher into Authenticated Encryption with Associated Data. As part of this work, we utilized an XOR procedure for constructing a generic tweakable cipher. Finally, we support two modes: nonce-respecting and nonce-misuse-resistant. Source code available at https://github.com/kenluck2001/cipherResearch/tree/main/src/tortoise.
Hi Model, generating 'nice' instead of 'good' is not as bad as generating 'rice'! Towards Context and Semantic Infused Dialogue Generation Loss Function and Evaluation Metric
Venue: arXiv cs.CL 2024
Year: 2024

Over the past two decades, dialogue modeling has made significant strides, moving from simple rule-based responses to personalized and persuasive response generation. However, despite these advancements, the objective functions and evaluation metrics for dialogue generation have remained stagnant. These lexical-based metrics, e.g., cross-entropy and BLEU, have two key limitations: (a) word-to-word matching without semantic consideration: It assigns the same credit for failure to generate "nice" and "rice" for "good", (b) missing context attribute for evaluating the generated response: Even if a generated response is relevant to the ongoing dialogue context, it may still be penalized for not matching the gold utterance provided in the corpus. In this paper, we first investigate these limitations comprehensively and propose a new loss function called Semantic Infused Contextualized diaLogue (SemTextualLogue) loss function. We also formulate an evaluation metric called Dialuation, incorporating both context and semantic relevance. We experimented with both non-pretrained and pre-trained models on two dialogue corpora, encompassing task-oriented and open-domain scenarios. We found that the dialogue generation models trained with SemTextualLogueloss attained superior performance compared to the traditional cross-entropy loss function. The findings establish that the effective training of a dialogue generation model hinges significantly on incorporating semantics and context. This pattern is also mirrored in the introduced Dialuation metric, where the consideration of both context and semantics correlates more strongly with human evaluation compared to traditional metrics.
Exploring Information Acquisition in Social Learning
Venue: arXiv cs.SI 2024
Year: 2024

Social learning, a fundamental process through which individuals shape their beliefs and perspectives via observation and interaction with others, is critical for the development of our society and the functioning of social governance. Prior works on social learning usually assume that the initial beliefs are given and focus on the update rule. With the recent proliferation of online social networks, there is an avalanche amount of information, which may significantly influence users' initial beliefs. In this paper, we use the rational inattention theory to model how agents acquire information to form initial beliefs and assess its influence on their adjustments in beliefs. Furthermore, we analyze the dynamic evolution of belief distribution among agents. Simulations and social experiments are conducted to validate our proposed model and analyze the impact of model parameters on belief dynamics.
Distributed Precoding for Satellite-Terrestrial Integrated Networks Without Sharing CSIT: A Rate-Splitting Approach
Venue: arXiv cs.IT 2024
Year: 2024

Satellite-terrestrial integrated networks (STINs) are promising architecture for providing global coverage. In STINs, full frequency reuse between a satellite and a terrestrial base station (BS) is encouraged for aggressive spectrum reuse, which induces non-negligible amount of interference. To address the interference management problem in STINs, this paper proposes a novel distributed precoding method. Key features of our method are: i) a rate-splitting (RS) strategy is incorporated for efficient interference management and ii) the precoders are designed in a distributed way without sharing channel state information between a satellite and a terrestrial BS. Specifically, to design the precoders in a distributed fashion, we put forth a spectral efficiency decoupling technique, that disentangles the total spectral efficiency function into two distinct terms, each of which is dependent solely on the satellite's precoder and the terrestrial BS's precoder, respectively. Then, to resolve the non-smoothness raised by the RS strategy, we approximate the spectral efficiency expression as a smooth function by using the LogSumExp technique; thereafter we develop a generalized power iteration inspired optimization algorithm built based on the first-order optimality condition. Simulation results demonstrate that the proposed method offers considerable spectral efficiency gains compared to the existing methods.
Non-smooth Control Barrier Functions for Stochastic Dynamical Systems
Venue: arXiv cs.RO 2024
Year: 2024

Uncertainties arising in various control systems, such as robots that are subject to unknown disturbances or environmental variations, pose significant challenges for ensuring system safety, such as collision avoidance. At the same time, safety specifications are getting more and more complex, e.g., by composing multiple safety objectives through Boolean operators resulting in non-smooth descriptions of safe sets. Control Barrier Functions (CBFs) have emerged as a control technique to provably guarantee system safety. In most settings, they rely on an assumption of having deterministic dynamics and smooth safe sets. This paper relaxes these two assumptions by extending CBFs to encompass control systems with stochastic dynamics and safe sets defined by non-smooth functions. By explicitly considering the stochastic nature of system dynamics and accommodating complex safety specifications, our method enables the design of safe control strategies in uncertain and complex systems. We provide formal guarantees on the safety of the system by leveraging the theoretical foundations of stochastic CBFs and non-smooth safe sets. Numerical simulations demonstrate the effectiveness of the approach in various scenarios.
A fixed-parameter tractable algorithm for combinatorial filter reduction
Venue: arXiv cs.RO 2024
Year: 2024

What is the minimal information that a robot must retain to achieve its task? To design economical robots, the literature dealing with reduction of combinatorial filters approaches this problem algorithmically. As lossless state compression is NP-hard, prior work has examined, along with minimization algorithms, a variety of special cases in which specific properties enable efficient solution. Complementing those findings, this paper refines the present understanding from the perspective of parameterized complexity. We give a fixed-parameter tractable algorithm for the general reduction problem by exploiting a transformation into clique covering. The transformation introduces new constraints that arise from sequential dependencies encoded within the input filter -- some of these constraints can be repaired, others are treated through enumeration. Through this approach, we identify parameters affecting filter reduction that are based upon inter-constraint couplings (expressed as a notion of their height and width), which add to the structural parameters present in the unconstrained problem of minimal clique covering. Compared with existing work, we precisely identify and quantitatively characterize those features that contribute to the problem's hardness: given a problem instance, the combinatorial core may be a fraction of the instance's full size, with a small subset of constraints needing to be considered, and even those may have directly identifiable couplings that collapse degrees of freedom in the enumeration.
Bayesian uncertainty-weighted loss for improved generalisability on polyp segmentation task
Venue: arXiv cs.CV 2024
Year: 2024

While several previous studies have devised methods for segmentation of polyps, most of these methods are not rigorously assessed on multi-center datasets. Variability due to appearance of polyps from one center to another, difference in endoscopic instrument grades, and acquisition quality result in methods with good performance on in-distribution test data, and poor performance on out-of-distribution or underrepresented samples. Unfair models have serious implications and pose a critical challenge to clinical applications. We adapt an implicit bias mitigation method which leverages Bayesian predictive uncertainties during training to encourage the model to focus on underrepresented sample regions. We demonstrate the potential of this approach to improve generalisability without sacrificing state-of-the-art performance on a challenging multi-center polyp segmentation dataset (PolypGen) with different centers and image modalities.
Multiplexed Streaming Codes for Messages with Different Decoding Delays in Channel with Burst and Random Erasures
Venue: arXiv cs.IT 2024
Year: 2024

In a real-time transmission scenario, messages are transmitted through a channel that is subject to packet loss. The destination must recover the messages within the required deadline. In this paper, we consider a setup where two different types of messages with distinct decoding deadlines are transmitted through a channel model that introduces either one burst erasure of length at most $B$, or $N$ random erasures in any fixed-sized sliding window. The message with a short decoding deadline $T_{\mathrm{u}}$ is referred to as an urgent message, while the other one with a decoding deadline $T_{\mathrm{v}}$ ($T_{\mathrm{v}} > T_{\mathrm{u}}$) is referred to as a less urgent message. We consider the scenario where $T_{\mathrm{v}} > T_{\mathrm{u}} + B$ and propose a non-trivial achievable region $\mathcal{R}$ for the aforementioned channel model. We propose a novel merging approach to encode two message streams of different urgency levels into a single flow and present explicit constructions for encoding, contributing to the establishment of the achievability of region $\mathcal{R}$. Our comprehensive analysis demonstrates that this region encompasses the rate pairs of existing encoding schemes and coincides with the capacity region in burst channel scenarios. Lastly, we investigate the property of the achievable region $\mathcal{R}$, proving that it is the largest one obtained from all the rate pairs under the merging method.
Multi-behavior Recommendation with SVD Graph Neural Networks
Venue: arXiv cs.IR 2024
Year: 2024

Graph Neural Networks (GNNs) have been extensively employed in the field of recommendation systems, offering users personalized recommendations and yielding remarkable outcomes. Recently, GNNs incorporating contrastive learning have demonstrated promising performance in handling the sparse data problem of recommendation systems. However, existing contrastive learning methods still have limitations in resisting noise interference, especially for multi-behavior recommendation. To mitigate the aforementioned issues, this paper proposes a GNN-based multi-behavior recommendation model called MB-SVD that utilizes Singular Value Decomposition (SVD) graphs to enhance model performance. In particular, MB-SVD considers user preferences across different behaviors, improving recommendation effectiveness. First, MB-SVD integrates the representation of users and items under different behaviors with learnable weight scores, which efficiently considers the influence of different behaviors. Then, MB-SVD generates augmented graph representation with global collaborative relations. Next, we simplify the contrastive learning framework by directly contrasting original representation with the enhanced representation using the InfoNCE loss. Through extensive experimentation, the remarkable performance of our proposed MB-SVD approach in multi-behavior recommendation endeavors across diverse real-world datasets is exhibited.
OYXOY: A Modern NLP Test Suite for Modern Greek
Venue: arXiv cs.CL 2024
Year: 2024

This paper serves as a foundational step towards the development of a linguistically motivated and technically relevant evaluation suite for Greek NLP. We initiate this endeavor by introducing four expert-verified evaluation tasks, specifically targeted at natural language inference, word sense disambiguation (through example comparison or sense selection) and metaphor detection. More than language-adapted replicas of existing tasks, we contribute two innovations which will resonate with the broader resource and evaluation community. Firstly, our inference dataset is the first of its kind, marking not just \textit{one}, but rather \textit{all} possible inference labels, accounting for possible shifts due to e.g. ambiguity or polysemy. Secondly, we demonstrate a cost-efficient method to obtain datasets for under-resourced languages. Using ChatGPT as a language-neutral parser, we transform the Dictionary of Standard Modern Greek into a structured format, from which we derive the other three tasks through simple projections. Alongside each task, we conduct experiments using currently available state of the art machinery. Our experimental baselines affirm the challenging nature of our tasks and highlight the need for expedited progress in order for the Greek NLP ecosystem to keep pace with contemporary mainstream research.
The Boundaries of Verifiable Accuracy, Robustness, and Generalisation in Deep Learning
Venue: arXiv cs.LG 2024
Year: 2024

In this work, we assess the theoretical limitations of determining guaranteed stability and accuracy of neural networks in classification tasks. We consider classical distribution-agnostic framework and algorithms minimising empirical risks and potentially subjected to some weights regularisation. We show that there is a large family of tasks for which computing and verifying ideal stable and accurate neural networks in the above settings is extremely challenging, if at all possible, even when such ideal solutions exist within the given class of neural architectures.