OpenRaiser/Intern-Atlas huggingface dataset:
Addressing the Elephant in the Room: Robust Animal Re-Identification with Unsupervised Part-Based Feature Alignment
Venue: arXiv cs.CV 2024
Year: 2024

Animal Re-ID is crucial for wildlife conservation, yet it faces unique challenges compared to person Re-ID. First, the scarcity and lack of diversity in datasets lead to background-biased models. Second, animal Re-ID depends on subtle, species-specific cues, further complicated by variations in pose, background, and lighting. This study addresses background biases by proposing a method to systematically remove backgrounds in both training and evaluation phases. And unlike prior works that depend on pose annotations, our approach utilizes an unsupervised technique for feature alignment across body parts and pose variations, enhancing practicality. Our method achieves superior results on three key animal Re-ID datasets: ATRW, YakReID-103, and ELPephants.
Efficient Two-Stage Gaussian Process Regression Via Automatic Kernel Search and Subsampling
Venue: arXiv cs.LG 2024
Year: 2024

Gaussian Process Regression (GPR) is widely used in statistics and machine learning for prediction tasks requiring uncertainty measures. Its efficacy depends on the appropriate specification of the mean function, covariance kernel function, and associated hyperparameters. Severe misspecifications can lead to inaccurate results and problematic consequences, especially in safety-critical applications. However, a systematic approach to handle these misspecifications is lacking in the literature. In this work, we propose a general framework to address these issues. Firstly, we introduce a flexible two-stage GPR framework that separates mean prediction and uncertainty quantification (UQ) to prevent mean misspecification, which can introduce bias into the model. Secondly, kernel function misspecification is addressed through a novel automatic kernel search algorithm, supported by theoretical analysis, that selects the optimal kernel from a candidate set. Additionally, we propose a subsampling-based warm-start strategy for hyperparameter initialization to improve efficiency and avoid hyperparameter misspecification. With much lower computational cost, our subsampling-based strategy can yield competitive or better performance than training exclusively on the full dataset. Combining all these components, we recommend two GPR methods-exact and scalable-designed to match available computational resources and specific UQ requirements. Extensive evaluation on real-world datasets, including UCI benchmarks and a safety-critical medical case study, demonstrates the robustness and precision of our methods.
Disentangle Sample Size and Initialization Effect on Perfect Generalization for Single-Neuron Target
Venue: arXiv cs.LG 2024
Year: 2024

Overparameterized models like deep neural networks have the intriguing ability to recover target functions with fewer sampled data points than parameters (see arXiv:2307.08921). To gain insights into this phenomenon, we concentrate on a single-neuron target recovery scenario, offering a systematic examination of how initialization and sample size influence the performance of two-layer neural networks. Our experiments reveal that a smaller initialization scale is associated with improved generalization, and we identify a critical quantity called the "initial imbalance ratio" that governs training dynamics and generalization under small initialization, supported by theoretical proofs. Additionally, we empirically delineate two critical thresholds in sample size--termed the "optimistic sample size" and the "separation sample size"--that align with the theoretical frameworks established by (see arXiv:2307.08921 and arXiv:2309.00508). Our results indicate a transition in the model's ability to recover the target function: below the optimistic sample size, recovery is unattainable; at the optimistic sample size, recovery becomes attainable albeit with a set of initialization of zero measure. Upon reaching the separation sample size, the set of initialization that can successfully recover the target function shifts from zero to positive measure. These insights, derived from a simplified context, provide a perspective on the intricate yet decipherable complexities of perfect generalization in overparameterized neural networks.
Guarding Multiple Secrets: Enhanced Summary Statistic Privacy for Data Sharing
Venue: arXiv cs.CR 2024
Year: 2024

Data sharing enables critical advances in many research areas and business applications, but it may lead to inadvertent disclosure of sensitive summary statistics (e.g., means or quantiles). Existing literature only focuses on protecting a single confidential quantity, while in practice, data sharing involves multiple sensitive statistics. We propose a novel framework to define, analyze, and protect multi-secret summary statistics privacy in data sharing. Specifically, we measure the privacy risk of any data release mechanism by the worst-case probability of an attacker successfully inferring summary statistic secrets. Given an attacker's objective spanning from inferring a subset to the entirety of summary statistic secrets, we systematically design and analyze tailored privacy metrics. Defining the distortion as the worst-case distance between the original and released data distribution, we analyze the tradeoff between privacy and distortion. Our contribution also includes designing and analyzing data release mechanisms tailored for different data distributions and secret types. Evaluations on real-world data demonstrate the effectiveness of our mechanisms in practical applications.
Predicting long time contributors with knowledge units of programming languages: an empirical study
Venue: arXiv cs.SE 2024
Year: 2024

Predicting potential long-time contributors (LTCs) early allows project maintainers to effectively allocate resources and mentoring to enhance their development and retention. Mapping programming language expertise to developers and characterizing projects in terms of how they use programming languages can help identify developers who are more likely to become LTCs. However, prior studies on predicting LTCs do not consider programming language skills. This paper reports an empirical study on the usage of knowledge units (KUs) of the Java programming language to predict LTCs. A KU is a cohesive set of key capabilities that are offered by one or more building blocks of a given programming language. We build a prediction model called KULTC, which leverages KU-based features along five different dimensions. We detect and analyze KUs from the studied 75 Java projects (353K commits and 168K pull requests) as well as 4,219 other Java projects in which the studied developers previously worked (1.7M commits). We compare the performance of KULTC with the state-of-the-art model, which we call BAOLTC. Even though KULTC focuses exclusively on the programming language perspective, KULTC achieves a median AUC of at least 0.75 and significantly outperforms BAOLTC. Combining the features of KULTC with the features of BAOLTC results in an enhanced model (KULTC+BAOLTC) that significantly outperforms BAOLTC with a normalized AUC improvement of 16.5%. Our feature importance analysis with SHAP reveals that developer expertise in the studied project is the most influential feature dimension for predicting LTCs. Finally, we develop a cost-effective model (KULTC_DEV_EXP+BAOLTC) that significantly outperforms BAOLTC. These encouraging results can be helpful to researchers who wish to further study the developers' engagement/retention to FLOSS projects or build models for predicting LTCs.
Implicit gaze research for XR systems
Venue: arXiv cs.HC 2024
Year: 2024

Although eye-tracking technology is being integrated into more VR and MR headsets, the true potential of eye tracking in enhancing user interactions within XR settings remains relatively untapped. Presently, one of the most prevalent gaze applications in XR is input control; for example, using gaze to control a cursor for pointing. However, our eyes evolved primarily for sensory input and understanding of the world around us, and yet few XR applications have leveraged natural gaze behavior to infer and support users' intent and cognitive states. Systems that can represent a user's context and interaction intent can better support the user by generating contextually relevant content, by making the user interface easier to use, by highlighting potential errors, and more. This mode of application is not fully taken advantage of in current commercially available XR systems and yet it is likely where we'll find paradigm-shifting use cases for eye tracking. In this paper, we elucidate the state-of-the-art applications for eye tracking and propose new research directions to harness its potential fully.
DeepNcode: Encoding-Based Protection against Bit-Flip Attacks on Neural Networks
Venue: arXiv cs.CR 2024
Year: 2024

Fault injection attacks are a potent threat against embedded implementations of neural network models. Several attack vectors have been proposed, such as misclassification, model extraction, and trojan/backdoor planting. Most of these attacks work by flipping bits in the memory where quantized model parameters are stored. In this paper, we introduce an encoding-based protection method against bit-flip attacks on neural networks, titled DeepNcode. We experimentally evaluate our proposal with several publicly available models and datasets, by using state-of-the-art bit-flip attacks: BFA, T-BFA, and TA-LBF. Our results show an increase in protection margin of up to $7.6\times$ for $4-$bit and $12.4\times$ for $8-$bit quantized networks. Memory overheads start at $50\%$ of the original network size, while the time overheads are negligible. Moreover, DeepNcode does not require retraining and does not change the original accuracy of the model.
Counting the number of inequivalent arithmetic expressions on $n$ variables
Venue: arXiv cs.DM 2024
Year: 2024

An expression is any mathematical formula that contains certain formal variables and operations to be executed in a specified order. In computer science, it is usually convenient to represent each expression in the form of an expression tree. Here, we consider only arithmetic expressions, i.e., those that contain only the four standard arithmetic operations: addition, subtraction, multiplication and division, alongside additive inversion. We first provide certain theoretical results concerning the equivalence of such expressions and then disclose a $Θ(n^2)$ algorithm that computes the number of inequivalent arithmetic expressions on $n$ distinct variables.
Leader Reward for POMO-Based Neural Combinatorial Optimization
Venue: arXiv cs.LG 2024
Year: 2024

Deep neural networks based on reinforcement learning (RL) for solving combinatorial optimization (CO) problems are developing rapidly and have shown a tendency to approach or even outperform traditional solvers. However, existing methods overlook an important distinction: CO problems differ from other traditional problems in that they focus solely on the optimal solution provided by the model within a specific length of time, rather than considering the overall quality of all solutions generated by the model. In this paper, we propose Leader Reward and apply it during two different training phases of the Policy Optimization with Multiple Optima (POMO) model to enhance the model's ability to generate optimal solutions. This approach is applicable to a variety of CO problems, such as the Traveling Salesman Problem (TSP), the Capacitated Vehicle Routing Problem (CVRP), and the Flexible Flow Shop Problem (FFSP), but also works well with other POMO-based models or inference phase's strategies. We demonstrate that Leader Reward greatly improves the quality of the optimal solutions generated by the model. Specifically, we reduce the POMO's gap to the optimum by more than 100 times on TSP100 with almost no additional computational overhead.
TaleMate: Exploring the use of Voice Agents for Parent-Child Joint Reading Experiences
Venue: arXiv cs.HC 2024
Year: 2024

Joint reading is a key activity for early learners, with caregiver-child interactions such as questioning and feedback playing an essential role in children's cognitive and linguistic development. However, for some parents, actively engaging children in storytelling can be challenging. To address this, we introduce TaleMate a platform designed to enhance shared reading by leveraging conversational agents that have been shown to support children's engagement and learning. TaleMate enables a dynamic, participatory reading experience where parents and children can choose which characters they wish to embody. Moreover, the system navigates the challenges posed by digital reading tools, such as decreased parent-child interaction, and builds upon the benefits of traditional and digital reading techniques. TaleMate offers an innovative approach to fostering early reading habits, bridging the gap between traditional joint reading practices and the digital reading landscape.
CIVICS: Building a Dataset for Examining Culturally-Informed Values in Large Language Models
Venue: arXiv cs.CL 2024
Year: 2024

This paper introduces the "CIVICS: Culturally-Informed & Values-Inclusive Corpus for Societal impacts" dataset, designed to evaluate the social and cultural variation of Large Language Models (LLMs) across multiple languages and value-sensitive topics. We create a hand-crafted, multilingual dataset of value-laden prompts which address specific socially sensitive topics, including LGBTQI rights, social welfare, immigration, disability rights, and surrogacy. CIVICS is designed to generate responses showing LLMs' encoded and implicit values. Through our dynamic annotation processes, tailored prompt design, and experiments, we investigate how open-weight LLMs respond to value-sensitive issues, exploring their behavior across diverse linguistic and cultural contexts. Using two experimental set-ups based on log-probabilities and long-form responses, we show social and cultural variability across different LLMs. Specifically, experiments involving long-form responses demonstrate that refusals are triggered disparately across models, but consistently and more frequently in English or translated statements. Moreover, specific topics and sources lead to more pronounced differences across model answers, particularly on immigration, LGBTQI rights, and social welfare. As shown by our experiments, the CIVICS dataset aims to serve as a tool for future research, promoting reproducibility and transparency across broader linguistic settings, and furthering the development of AI technologies that respect and reflect global cultural diversities and value pluralism. The CIVICS dataset and tools will be made available upon publication under open licenses; an anonymized version is currently available at https://huggingface.co/CIVICS-dataset.
Towards A Comprehensive Assessment of AI's Environmental Impact
Venue: arXiv cs.CY 2024
Year: 2024

Artificial Intelligence, machine learning (AI/ML) has allowed exploring solutions for a variety of environmental and climate questions ranging from natural disasters, greenhouse gas emission, monitoring biodiversity, agriculture, to weather and climate modeling, enabling progress towards climate change mitigation. However, the intersection of AI/ML and environment is not always positive. The recent surge of interest in ML, made possible by processing very large volumes of data, fueled by access to massive compute power, has sparked a trend towards large-scale adoption of AI/ML. This interest places tremendous pressure on natural resources, that are often overlooked and under-reported. There is a need for a framework that monitors the environmental impact and degradation from AI/ML throughout its lifecycle for informing policymakers, stakeholders to adequately implement standards and policies and track the policy outcome over time. For these policies to be effective, AI's environmental impact needs to be monitored in a spatially-disaggregated, timely manner across the globe at the key activity sites. This study proposes a methodology to track environmental variables relating to the multifaceted impact of AI around datacenters using openly available energy data and globally acquired satellite observations. We present a case study around Northern Virginia, United States that hosts a growing number of datacenters and observe changes in multiple satellite-based environmental metrics. We then discuss the steps to expand this methodology for comprehensive assessment of AI's environmental impact across the planet. We also identify data gaps and formulate recommendations for improving the understanding and monitoring AI-induced changes to the environment and climate.
Evaluating Large Language Models with Human Feedback: Establishing a Swedish Benchmark
Venue: arXiv cs.CL 2024
Year: 2024

In the rapidly evolving field of artificial intelligence, large language models (LLMs) have demonstrated significant capabilities across numerous applications. However, the performance of these models in languages with fewer resources, such as Swedish, remains under-explored. This study introduces a comprehensive human benchmark to assess the efficacy of prominent LLMs in understanding and generating Swedish language texts using forced choice ranking. We employ a modified version of the ChatbotArena benchmark, incorporating human feedback to evaluate eleven different models, including GPT-4, GPT-3.5, various Claude and Llama models, and bespoke models like Dolphin-2.9-llama3b-8b-flashback and BeagleCatMunin. These models were chosen based on their performance on LMSYS chatbot arena and the Scandeval benchmarks. We release the chatbotarena.se benchmark as a tool to improve our understanding of language model performance in Swedish with the hopes that it will be widely used. We aim to create a leaderboard once sufficient data has been collected and analysed.
Towards a Unified Framework for Evaluating Explanations
Venue: arXiv cs.LG 2024
Year: 2024

The challenge of creating interpretable models has been taken up by two main research communities: ML researchers primarily focused on lower-level explainability methods that suit the needs of engineers, and HCI researchers who have more heavily emphasized user-centered approaches often based on participatory design methods. This paper reviews how these communities have evaluated interpretability, identifying overlaps and semantic misalignments. We propose moving towards a unified framework of evaluation criteria and lay the groundwork for such a framework by articulating the relationships between existing criteria. We argue that explanations serve as mediators between models and stakeholders, whether for intrinsically interpretable models or opaque black-box models analyzed via post-hoc techniques. We further argue that useful explanations require both faithfulness and intelligibility. Explanation plausibility is a prerequisite for intelligibility, while stability is a prerequisite for explanation faithfulness. We illustrate these criteria, as well as specific evaluation methods, using examples from an ongoing study of an interpretable neural network for predicting a particular learner behavior.
Reverse Engineering Structure and Semantics of Input of a Binary Executable
Venue: arXiv cs.CR 2024
Year: 2024

Knowledge of the input format of binary executables is important for finding bugs and vulnerabilities, such as generating data for fuzzing or manual reverse engineering. This paper presents an algorithm to recover the structure and semantic relations between fields of the input of binary executables using dynamic taint analysis. The algorithm improves upon prior work by not just partitioning the input into consecutive bytes representing values but also identifying syntactic components of structures, such as atomic fields of fixed and variable lengths, and different types of arrays, such as arrays of atomic fields, arrays of records, and arrays with variant records. It also infers the semantic relations between fields of a structure, such as count fields that specify the count of an array of records or offset fields that specify the start location of a variable-length field within the input data. The algorithm constructs a C/C++-like structure to represent the syntactic components and semantic relations. The algorithm was implemented in a prototype system named ByteRI 2.0. The system was evaluated using a controlled experiment with synthetic subject programs and real-world programs. The subject programs were created to accept a variety of input formats that mimic syntactic components and selected semantic relations found in conventional data formats, such as PE, PNG, ZIP, and CSV. The results show that ByteRI 2.0 correctly identifies the syntactic elements and their grammatical structure, as well as the semantic relations between the fields for both synthetic subject programs and real-world programs. The recovered structures, when used as a generator, produced valid data that was acceptable for all the synthetic subject programs and some of the real-world programs.
On the Role of Non-Terrestrial Networks for Boosting Terrestrial Network Performance in Dynamic Traffic Scenarios
Venue: arXiv cs.NI 2024
Year: 2024

Due to an ever-expansive network deployment, numerous questions are being raised regarding the energy consumption of the mobile network. Recently, Non-Terrestrial Networks (NTNs) have proven to be a useful, and complementary solution to Terrestrial Networks (TN) to provide ubiquitous coverage. In this paper, we consider an integrated TN-NTN, and study how to maximize its resource usage in a dynamic traffic scenario. We introduce BLASTER, a framework designed to control User Equipment (UE) association, Base Station (BS) transmit power and activation, and bandwidth allocation between the terrestrial and non-terrestrial tiers. Our proposal is able to adapt to fluctuating daily traffic, focusing on reducing power consumption throughout the network during low traffic and distributing the load otherwise. Simulation results show an average daily decrease of total power consumption by 45% compared to a network model following 3GPP recommendation, as well as an average throughput increase of roughly 250%. Our paper underlines the central and dynamic role that the NTN plays in improving key areas of concern for network flexibility.
Attending to Topological Spaces: The Cellular Transformer
Venue: arXiv cs.LG 2024
Year: 2024

Topological Deep Learning seeks to enhance the predictive performance of neural network models by harnessing topological structures in input data. Topological neural networks operate on spaces such as cell complexes and hypergraphs, that can be seen as generalizations of graphs. In this work, we introduce the Cellular Transformer (CT), a novel architecture that generalizes graph-based transformers to cell complexes. First, we propose a new formulation of the usual self- and cross-attention mechanisms, tailored to leverage incidence relations in cell complexes, e.g., edge-face and node-edge relations. Additionally, we propose a set of topological positional encodings specifically designed for cell complexes. By transforming three graph datasets into cell complex datasets, our experiments reveal that CT not only achieves state-of-the-art performance, but it does so without the need for more complex enhancements such as virtual nodes, in-domain structural encodings, or graph rewiring.