OpenRaiser/Intern-Atlas huggingface dataset:
A Joint Code and Belief Propagation Decoder Design for Quantum LDPC Codes
Venue: arXiv cs.IT 2024
Year: 2024

Quantum low-density parity-check (QLDPC) codes are among the most promising candidates for future quantum error correction schemes. However, a limited number of short to moderate-length QLDPC codes have been designed and their decoding performance is sub-optimal with a quaternary belief propagation (BP) decoder due to unavoidable short cycles in their Tanner graphs. In this paper, we propose a novel joint code and decoder design for QLDPC codes. The constructed codes have a minimum distance of about the square root of the block length. In addition, it is, to the best of our knowledge, the first QLDPC code family where BP decoding is not impaired by short cycles of length 4. This is achieved by using an ensemble BP decoder mitigating the influence of assembled short cycles. We outline two code construction methods based on classical quasi-cyclic codes and finite geometry codes. Numerical results demonstrate outstanding decoding performance over depolarizing channels.
Knowledge-Centric Templatic Views of Documents
Venue: arXiv cs.CL 2024
Year: 2024

Authors seeking to communicate with broader audiences often share their ideas in various document formats, such as slide decks, newsletters, reports, and posters. Prior work on document generation has generally tackled the creation of each separate format to be a different task, leading to fragmented learning processes, redundancy in models and methods, and disjointed evaluation. We consider each of these documents as templatic views of the same underlying knowledge/content, and we aim to unify the generation and evaluation of these templatic views. We begin by showing that current LLMs are capable of generating various document formats with little to no supervision. Further, a simple augmentation involving a structured intermediate representation can improve performance, especially for smaller models. We then introduce a novel unified evaluation framework that can be adapted to measuring the quality of document generators for heterogeneous downstream applications. This evaluation is adaptable to a range of user defined criteria and application scenarios, obviating the need for task specific evaluation metrics. Finally, we conduct a human evaluation, which shows that people prefer 82% of the documents generated with our method, while correlating more highly with our unified evaluation framework than prior metrics in the literature.
Transformer for Object Re-Identification: A Survey
Venue: arXiv cs.CV 2024
Year: 2024

Object Re-identification (Re-ID) aims to identify specific objects across different times and scenes, which is a widely researched task in computer vision. For a prolonged period, this field has been predominantly driven by deep learning technology based on convolutional neural networks. In recent years, the emergence of Vision Transformers has spurred a growing number of studies delving deeper into Transformer-based Re-ID, continuously breaking performance records and witnessing significant progress in the Re-ID field. Offering a powerful, flexible, and unified solution, Transformers cater to a wide array of Re-ID tasks with unparalleled efficacy. This paper provides a comprehensive review and in-depth analysis of the Transformer-based Re-ID. In categorizing existing works into Image/Video-Based Re-ID, Re-ID with limited data/annotations, Cross-Modal Re-ID, and Special Re-ID Scenarios, we thoroughly elucidate the advantages demonstrated by the Transformer in addressing a multitude of challenges across these domains. Considering the trending unsupervised Re-ID, we propose a new Transformer baseline, UntransReID, achieving state-of-the-art performance on both single/cross modal tasks. For the under-explored animal Re-ID, we devise a standardized experimental benchmark and conduct extensive experiments to explore the applicability of Transformer for this task and facilitate future research. Finally, we discuss some important yet under-investigated open issues in the large foundation model era, we believe it will serve as a new handbook for researchers in this field. A periodically updated website will be available at https://github.com/mangye16/ReID-Survey.
CHAMP: A Competition-level Dataset for Fine-Grained Analyses of LLMs' Mathematical Reasoning Capabilities
Venue: arXiv cs.CL 2024
Year: 2024

Recent large language models (LLMs) have shown indications of mathematical reasoning ability on challenging competition-level problems, especially with self-generated verbalizations of intermediate reasoning steps (i.e., chain-of-thought prompting). However, current evaluations mainly focus on the end-to-end final answer correctness, and it is unclear whether LLMs can make use of helpful side information such as problem-specific hints. In this paper, we propose a challenging benchmark dataset for enabling such analyses. The Concept and Hint-Annotated Math Problems (CHAMP) consists of high school math competition problems, annotated with concepts, or general math facts, and hints, or problem-specific tricks. These annotations allow us to explore the effects of additional information, such as relevant hints, misleading concepts, or related problems. This benchmark is difficult, with the best model only scoring 58.1% in standard settings. With concepts and hints, performance sometimes improves, indicating that some models can make use of such side information. Furthermore, we annotate model-generated solutions for their correctness. Using this corpus, we find that models often arrive at the correct final answer through wrong reasoning steps. In addition, we test whether models are able to verify these solutions, and find that most models struggle.
A metric for characterizing the arm nonuse workspace in poststroke individuals using a robot arm
Venue: arXiv cs.RO 2024
Year: 2024

An over-reliance on the less-affected limb for functional tasks at the expense of the paretic limb and in spite of recovered capacity is an often-observed phenomenon in survivors of hemispheric stroke. The difference between capacity for use and actual spontaneous use is referred to as arm nonuse. Obtaining an ecologically valid evaluation of arm nonuse is challenging because it requires the observation of spontaneous arm choice for different tasks, which can easily be influenced by instructions, presumed expectations, and awareness that one is being tested. To better quantify arm nonuse, we developed the Bimanual Arm Reaching Test with a Robot (BARTR) for quantitatively assessing arm nonuse in chronic stroke survivors. The BARTR is an instrument that utilizes a robot arm as a means of remote and unbiased data collection of nuanced spatial data for clinical evaluations of arm nonuse. This approach shows promise for determining the efficacy of interventions designed to reduce paretic arm nonuse and enhance functional recovery after stroke. We show that the BARTR satisfies the criteria of an appropriate metric for neurorehabilitative contexts: it is valid, reliable, and simple to use.
Towards Effective Image Forensics via A Novel Computationally Efficient Framework and A New Image Splice Dataset
Venue: arXiv cs.CV 2024
Year: 2024

Splice detection models are the need of the hour since splice manipulations can be used to mislead, spread rumors and create disharmony in society. However, there is a severe lack of image splicing datasets, which restricts the capabilities of deep learning models to extract discriminative features without overfitting. This manuscript presents two-fold contributions toward splice detection. Firstly, a novel splice detection dataset is proposed having two variants. The two variants include spliced samples generated from code and through manual editing. Spliced images in both variants have corresponding binary masks to aid localization approaches. Secondly, a novel Spatio-Compression Lightweight Splice Detection Framework is proposed for accurate splice detection with minimum computational cost. The proposed dual-branch framework extracts discriminative spatial features from a lightweight spatial branch. It uses original resolution compression data to extract double compression artifacts from the second branch, thereby making it 'information preserving.' Several CNNs are tested in combination with the proposed framework on a composite dataset of images from the proposed dataset and the CASIA v2.0 dataset. The best model accuracy of 0.9382 is achieved and compared with similar state-of-the-art methods, demonstrating the superiority of the proposed framework.
Joint Extraction of Uyghur Medicine Knowledge with Edge Computing
Venue: arXiv cs.CL 2024
Year: 2024

Medical knowledge extraction methods based on edge computing deploy deep learning models on edge devices to achieve localized entity and relation extraction. This approach avoids transferring substantial sensitive data to cloud data centers, effectively safeguarding the privacy of healthcare services. However, existing relation extraction methods mainly employ a sequential pipeline approach, which classifies relations between determined entities after entity recognition. This mode faces challenges such as error propagation between tasks, insufficient consideration of dependencies between the two subtasks, and the neglect of interrelations between different relations within a sentence. To address these challenges, a joint extraction model with parameter sharing in edge computing is proposed, named CoEx-Bert. This model leverages shared parameterization between two models to jointly extract entities and relations. Specifically, CoEx-Bert employs two models, each separately sharing hidden layer parameters, and combines these two loss functions for joint backpropagation to optimize the model parameters. Additionally, it effectively resolves the issue of entity overlapping when extracting knowledge from unstructured Uyghur medical texts by considering contextual relations. Finally, this model is deployed on edge devices for real-time extraction and inference of Uyghur medical knowledge. Experimental results demonstrate that CoEx-Bert outperforms existing state-of-the-art methods, achieving accuracy, recall, and F1 scores of 90.65\%, 92.45\%, and 91.54\%, respectively, in the Uyghur traditional medical literature dataset. These improvements represent a 6.45\% increase in accuracy, a 9.45\% increase in recall, and a 7.95\% increase in F1 score compared to the baseline.
GEML: A Grammar-based Evolutionary Machine Learning Approach for Design-Pattern Detection
Venue: arXiv cs.SE 2024
Year: 2024

Design patterns (DPs) are recognised as a good practice in software development. However, the lack of appropriate documentation often hampers traceability, and their benefits are blurred among thousands of lines of code. Automatic methods for DP detection have become relevant but are usually based on the rigid analysis of either software metrics or specific properties of the source code. We propose GEML, a novel detection approach based on evolutionary machine learning using software properties of diverse nature. Firstly, GEML makes use of an evolutionary algorithm to extract those characteristics that better describe the DP, formulated in terms of human-readable rules, whose syntax is conformant with a context-free grammar. Secondly, a rule-based classifier is built to predict whether new code contains a hidden DP implementation. GEML has been validated over five DPs taken from a public repository recurrently adopted by machine learning studies. Then, we increase this number up to 15 diverse DPs, showing its effectiveness and robustness in terms of detection capability. An initial parameter study served to tune a parameter setup whose performance guarantees the general applicability of this approach without the need to adjust complex parameters to a specific pattern. Finally, a demonstration tool is also provided.
Detachment Problem -- Application in Prevention of Information Leakage in Stock Markets
Venue: arXiv cs.SI 2024
Year: 2024

In this paper, we introduce the Detachment Problem. It can be seen as a generalized Vaccination Problem. The aim is to optimally cut the individuals' ties to circles that connect them to others, to minimize the overall information transfer in a social network. When an individual is isolated from a particular circle, it leads to the elimination of the connections to all the members of that circle, yet the connections to other circles remain. This approach contrasts with the conventional vaccination problem, in which a subset of vertices is totally eliminated. In our case, the connections of individuals to their circles are selectively, rather than entirely, eliminated. Contextually, this article focuses on private information flows, specifically within networks formed by memberships in circles of insiders in companies. Our quasi-empirical study uses simulated information flows on an observable network, and the statistical properties of the simulated information flows are matched with real-world data. In a broader context, this paper presents the Detachment Problem as a versatile approach for optimal social distancing, applicable across various scenarios. We propose and define a concept of expected proportional outside influence, or EPOI, as measure of how widespread information leak is. We also implement a greedy algorithm for finding a set of detachments to minimize EPOI. For comparison, we devise a simple heuristic based on minimal cut, to separate the most influential circles from each other. We provide evidence that the greedy algorithm is not optimal, and it is sometimes outperformed by the simple heuristic minimum cut algorithm, However, the greedy algorithm outperforms the cut algorithm in most cases. Further avenues of research are discussed.
Exploring Adversarial Attacks against Latent Diffusion Model from the Perspective of Adversarial Transferability
Venue: arXiv cs.CV 2024
Year: 2024

Recently, many studies utilized adversarial examples (AEs) to raise the cost of malicious image editing and copyright violation powered by latent diffusion models (LDMs). Despite their successes, a few have studied the surrogate model they used to generate AEs. In this paper, from the perspective of adversarial transferability, we investigate how the surrogate model's property influences the performance of AEs for LDMs. Specifically, we view the time-step sampling in the Monte-Carlo-based (MC-based) adversarial attack as selecting surrogate models. We find that the smoothness of surrogate models at different time steps differs, and we substantially improve the performance of the MC-based AEs by selecting smoother surrogate models. In the light of the theoretical framework on adversarial transferability in image classification, we also conduct a theoretical analysis to explain why smooth surrogate models can also boost AEs for LDMs.
Evolving Code with A Large Language Model
Venue: arXiv cs.NE 2024
Year: 2024

Algorithms that use Large Language Models (LLMs) to evolve code arrived on the Genetic Programming (GP) scene very recently. We present LLM GP, a formalized LLM-based evolutionary algorithm designed to evolve code. Like GP, it uses evolutionary operators, but its designs and implementations of those operators radically differ from GP's because they enlist an LLM, using prompting and the LLM's pre-trained pattern matching and sequence completion capability. We also present a demonstration-level variant of LLM GP and share its code. By addressing algorithms that range from the formal to hands-on, we cover design and LLM-usage considerations as well as the scientific challenges that arise when using an LLM for genetic programming.
Directed Regular and Context-Free Languages
Venue: arXiv cs.FL 2024
Year: 2024

We study the problem of deciding whether a given language is directed. A language $L$ is \emph{directed} if every pair of words in $L$ have a common (scattered) superword in $L$. Deciding directedness is a fundamental problem in connection with ideal decompositions of downward closed sets. Another motivation is that deciding whether two \emph{directed} context-free languages have the same downward closures can be decided in polynomial time, whereas for general context-free languages, this problem is known to be coNEXP-complete. We show that the directedness problem for regular languages, given as NFAs, belongs to $AC^1$, and thus polynomial time. Moreover, it is NL-complete for fixed alphabet sizes. Furthermore, we show that for context-free languages, the directedness problem is PSPACE-complete.
Curator: Efficient Indexing for Multi-Tenant Vector Databases
Venue: arXiv cs.DB 2024
Year: 2024

Vector databases have emerged as key enablers for bridging intelligent applications with unstructured data, providing generic search and management support for embedding vectors extracted from the raw unstructured data. As multiple data users can share the same database infrastructure, multi-tenancy support for vector databases is increasingly desirable. This hinges on an efficient filtered search operation, i.e., only querying the vectors accessible to a particular tenant. Multi-tenancy in vector databases is currently achieved by building either a single, shared index among all tenants, or a per-tenant index. The former optimizes for memory efficiency at the expense of search performance, while the latter does the opposite. Instead, this paper presents Curator, an in-memory vector index design tailored for multi-tenant queries that simultaneously achieves the two conflicting goals, low memory overhead and high performance for queries, vector insertion, and deletion. Curator indexes each tenant's vectors with a tenant-specific clustering tree and encodes these trees compactly as sub-trees of a shared clustering tree. Each tenant's clustering tree adapts dynamically to its unique vector distribution, while maintaining a low per-tenant memory footprint. Our evaluation, based on two widely used data sets, confirms that Curator delivers search performance on par with per-tenant indexing, while maintaining memory consumption at the same level as metadata filtering on a single, shared index.
Secrecy Coding for the Binary Symmetric Wiretap Channel via Linear Programming
Venue: arXiv cs.IT 2024
Year: 2024

In this paper, we use a linear programming (LP) optimization approach to evaluate the equivocation for a wiretap channel where the main channel is noiseless, and the wiretap channel is a binary symmetric channel (BSC). Using this technique, we present an analytical limit for the achievable secrecy rate in the finite blocklength regime that is tighter than traditional fundamental limits. We also propose a secrecy coding technique that outperforms random binning codes. When there is one overhead bit, this coding technique is optimum and achieves the analytical limit. For cases with additional bits of overhead, our coding scheme can achieve equivocation rates close to the new limit. Furthermore, we evaluate the patterns of the generator matrix and the parity-check matrix for linear codes and we present binning techniques for both linear and non-linear codes using two different approaches: recursive and non-recursive. To our knowledge, this is the first optimization solution for secrecy coding obtained through linear programming.
CAC 2.0: A Corrupt and Correct Logic Locking Technique Resilient to Structural Analysis Attacks
Venue: arXiv cs.CR 2024
Year: 2024

Logic locking proposed to protect integrated circuits from serious hardware threats has been studied extensively over a decade. In these years, many efficient logic locking techniques have been proven to be broken. The state-of-the-art logic locking techniques, including the prominent corrupt and correct (CAC) technique, are resilient to satisfiability (SAT)-based and removal attacks, but vulnerable to structural analysis attacks. To overcome this drawback, this paper introduces an improved version of CAC, called CAC 2.0, which increases the search space of structural analysis attacks using obfuscation. To do so, CAC 2.0 locks the original circuit twice, one after another, on different nodes with different number of protected primary inputs using CAC, while hiding original protected primary inputs among decoy primary inputs. This paper also introduces an open source logic locking tool, called HIID, equipped with well-known techniques including CAC 2.0. Our experiments show that CAC 2.0 is resilient to existing SAT-based, removal, and structural analysis attacks. To achieve this, it increases the number of key inputs at most 4x and the gate-level area between 30.2% and 0.8% on circuits with low and high complexity with respect to CAC.
Adaptive Prognostic Malfunction Based Processor for Autonomous Landing Guidance Assistance System Using FPGA
Venue: arXiv cs.AR 2024
Year: 2024

The demand for more developed and agile urban taxi drones is increasing rapidly nowadays to sustain crowded cities and their traffic issues. The critical factor for spreading such technology could be related to the safety criteria that must be considered. One of the most critical safety aspects for such vertical and/or Short Take-Off and Landing (V/STOL) drones is related to safety during the landing stage, in which most of the recent flight accidents have occurred. This paper focused on solving this issue by proposing decentralized processing cores that could improve the landing failure rate by depending on a Fuzzy Logic System (FLS) and additional Digital Signal Processing (DSP) elements. Also, the proposed system will enhance the safety factor during the landing stages by adding a self-awareness feature in case a certain sensor malfunction occurs using the proposed Adaptive Prognostic Malfunction Unit (APMU). This proposed coarse-grained Autonomous Landing Guidance Assistance System (ALGAS4) processing architecture has been optimized using different optimization techniques. The ALGAS4 architecture has been designed completely using VHDL, and the targeted FPGA was the INTEL Cyclone V 5CGXFC9D6F27C7 chip. According to the synthesis findings of the INTEL Quartus Prime software, the maximum working frequency of the ALGAS4 system is 278.24 MHz. In addition, the proposed ALGAS4 system could maintain a maximum computing performance of approximately 74.85 GOPS while using just 166.56 mW for dynamic and I/O power dissipation.
Inroads to a Structured Data Natural Language Bijection and the role of LLM annotation
Venue: arXiv cs.CL 2024
Year: 2024

This work finds limited evidence supporting the theory that using multiple tasks with sequence-to-sequence transformer language models can improve performance on some metrics. In particular, the multi-task generalist t5-small outperforms the specialist t5-small with a $F_1$ of $0.771$ up from $0.692$, which may point to underlying cross-task knowledge generalization. This further suggests that even with the same network, "re-using" the same data in a different way may lead to higher performance in some metrics. However, the inverse task alone is likely only an optimization strategy, since it does not yield a significant general improvement at the model sizes explored in this work. Also, adding $\approx 4500$ LLM annotated records (interlaced with the $12800$ WebNLG training records) does not substantially change automatic metric performance compared to the same t5-small model without the synthetic data. This may be due to a learning capacity bottleneck on account of model size, and decreases observed may be due to distributional differences in the corpora. Future research using larger models or human evaluation is required to more fully explain the mechanisms contributing to performance on these tasks.