OpenRaiser/Intern-Atlas huggingface dataset:
Region-specific Risk Quantification for Interpretable Prognosis of COVID-19
Venue: arXiv cs.CV 2024
Year: 2024

The COVID-19 pandemic has strained global public health, necessitating accurate diagnosis and intervention to control disease spread and reduce mortality rates. This paper introduces an interpretable deep survival prediction model designed specifically for improved understanding and trust in COVID-19 prognosis using chest X-ray (CXR) images. By integrating a large-scale pretrained image encoder, Risk-specific Grad-CAM, and anatomical region detection techniques, our approach produces regional interpretable outcomes that effectively capture essential disease features while focusing on rare but critical abnormal regions. Our model's predictive results provide enhanced clarity and transparency through risk area localization, enabling clinicians to make informed decisions regarding COVID-19 diagnosis with better understanding of prognostic insights. We evaluate the proposed method on a multi-center survival dataset and demonstrate its effectiveness via quantitative and qualitative assessments, achieving superior C-indexes (0.764 and 0.727) and time-dependent AUCs (0.799 and 0.691). These results suggest that our explainable deep survival prediction model surpasses traditional survival analysis methods in risk prediction, improving interpretability for clinical decision making and enhancing AI system trustworthiness.
Labeling supervised fine-tuning data with the scaling law
Venue: arXiv cs.CL 2024
Year: 2024

This paper introduces a multi-stage manual annotation calibrated by the scaling law, offering a high-quality Supervised Fine-Tuning data acquisition method for environments with constrained resources like GPU poor, limited GPT access, and funding restrictions. We have preprocessed 58k authentic chat data and manually annotated 2.3k questions. After this, we conducted fine-tuning on Qwen models, ranging from 0.5B to 32B parameters. The optimal version improved 29.07 in F1 score. This confirms the viability of fine-tuning Large Language Model (LLM) for downstream Natural Language Processing (NLP) tasks. Our contributions are: 1) Created Supervised Fine-Tuning (SFT) training data in alpaca format, along with a set of Low-Rank Adaptation (LoRA) weights, and 2) Developed a method for acquiring high-quality data leveraging scaling law principle. The script, raw data with alpaca format and experiments track are open-sourced on Github (https://github.com/InternLM/HuixiangDou/tree/main/web/tools), HuggingFace (https://huggingface.co/tpoisonooo) and WandB (https://wandb.ai/tpoisonooo/huixiangdou-cr/table?nw=nwusertpoisonooo). The privacy of the data involved has been authorized by users. SFT data and license comes from ncnn contributors group.
Site-Specific Deployment Optimization of Intelligent Reflecting Surface for Coverage Enhancement
Venue: arXiv cs.IT 2024
Year: 2024

Intelligent Reflecting Surface (IRS) is a promising technology for next generation wireless networks. Despite substantial research in IRS-aided communications, the assumed antenna and channel models are typically simplified without considering site-specific characteristics, which in turn critically affect the IRS deployment and performance in a given environment. In this paper, we first investigate the link-level performance of active or passive IRS taking into account the IRS element radiation pattern (ERP) as well as the antenna radiation pattern of the access point (AP). Then the network-level coverage performance is evaluated/optimized in site-specific multi-building scenarios, by properly deploying multiple IRSs on candidate building facets to serve a given set of users or Points of Interests (PoIs). The problem is reduced to an integer linear programming (ILP) based on given link-level metrics, which is then solved efficiently under moderate network sizes. Numerical results confirm the impact of AP antenna/IRS element pattern on the link-level performance. In addition, it is found that active IRSs, though associated with higher hardware complexity and cost, significantly improve the site-specific network coverage performance in terms of average ergodic rate and fairness among the PoIs as well as the range of serving area, compared with passive IRSs that have a much larger number of elements.
An FPT Algorithm for the Exact Matching Problem and NP-hardness of Related Problems
Venue: arXiv cs.DS 2024
Year: 2024

The exact matching problem is a constrained variant of the maximum matching problem: given a graph with each edge having a weight $0$ or $1$ and an integer $k$, the goal is to find a perfect matching of weight exactly $k$. Mulmuley, Vazirani, and Vazirani (1987) proposed a randomized polynomial-time algorithm for this problem, and it is still open whether it can be derandomized. Very recently, El Maalouly, Steiner, and Wulf (2023) showed that for bipartite graphs there exists a deterministic FPT algorithm parameterized by the (bipartite) independence number. In this paper, by extending a part of their work, we propose a deterministic FPT algorithm in general parameterized by the minimum size of an odd cycle transversal in addition to the (bipartite) independence number. We also consider a relaxed problem called the correct parity matching problem, and show that a slight generalization of an equivalent problem is NP-hard.
Responsible AI: Portraits with Intelligent Bibliometrics
Venue: arXiv cs.AI 2024
Year: 2024

Shifting the focus from principles to practical implementation, responsible artificial intelligence (AI) has garnered considerable attention across academia, industry, and society at large. Despite being in its nascent stages, this emerging field grapples with nebulous concepts and intricate knowledge frameworks. By analyzing three prevailing concepts - explainable AI, trustworthy AI, and ethical AI, this study defined responsible AI and identified its core principles. Methodologically, this study successfully demonstrated the implementation of leveraging AI's capabilities into bibliometrics for enhanced knowledge discovery and the cross-validation of experimentally examined models with domain insights. Empirically, this study investigated 17,799 research articles contributed by the AI community since 2015. This involves recognizing key technological players and their relationships, unveiling the topical landscape and hierarchy of responsible AI, charting its evolution, and elucidating the interplay between the responsibility principles and primary AI techniques. An analysis of a core cohort comprising 380 articles from multiple disciplines captures the most recent advancements in responsible AI. As one of the pioneering bibliometric studies dedicated to exploring responsible AI, this study will provide comprehensive macro-level insights, enhancing the understanding of responsible AI while furnishing valuable knowledge support for AI regulation and governance initiatives.
Insights Gained after a Decade of Cellular Automata-based Cryptography
Venue: arXiv cs.CR 2024
Year: 2024

Cellular Automata (CA) have been extensively used to implement symmetric cryptographic primitives, such as pseudorandom number generators and S-boxes. However, most of the research in this field, except the very early works, seems to be published in non-cryptographic venues. This phenomenon poses a problem of relevance: are CA of any use to cryptographers nowadays? This paper provides insights into this question by briefly outlining the history of CA-based cryptography. In doing so, the paper identifies some shortcomings in the research addressing the design of symmetric primitives exclusively from a CA standpoint, alongside some recommendations for future research. Notably, the paper remarks that researchers working in CA and cryptography often tackle similar problems, albeit under different perspectives and terminologies. This observation indicates that there is still ample room for fruitful collaborations between the CA and cryptography communities in the future.
Predicting Open-Hole Laminates Failure Using Support Vector Machines With Classical and Quantum Kernels
Venue: arXiv cs.CE 2024
Year: 2024

Modeling open hole failure of composites is a complex task, consisting in a highly nonlinear response with interacting failure modes. Numerical modeling of this phenomenon has traditionally been based on the finite element method, but requires to tradeoff between high fidelity and computational cost. To mitigate this shortcoming, recent work has leveraged machine learning to predict the strength of open hole composite specimens. Here, we also propose using data-based models but to tackle open hole composite failure from a classification point of view. More specifically, we show how to train surrogate models to learn the ultimate failure envelope of an open hole composite plate under in-plane loading. To achieve this, we solve the classification problem via support vector machine (SVM) and test different classifiers by changing the SVM kernel function. The flexibility of kernel-based SVM also allows us to integrate the recently developed quantum kernels in our algorithm and compare them with the standard radial basis function (RBF) kernel. Finally, thanks to kernel-target alignment optimization, we tune the free parameters of all kernels to best separate safe and failure-inducing loading states. The results show classification accuracies higher than 90% for RBF, especially after alignment, followed closely by the quantum kernel classifiers.
RepAugment: Input-Agnostic Representation-Level Augmentation for Respiratory Sound Classification
Venue: arXiv cs.SD 2024
Year: 2024

Recent advancements in AI have democratized its deployment as a healthcare assistant. While pretrained models from large-scale visual and audio datasets have demonstrably generalized to this task, surprisingly, no studies have explored pretrained speech models, which, as human-originated sounds, intuitively would share closer resemblance to lung sounds. This paper explores the efficacy of pretrained speech models for respiratory sound classification. We find that there is a characterization gap between speech and lung sound samples, and to bridge this gap, data augmentation is essential. However, the most widely used augmentation technique for audio and speech, SpecAugment, requires 2-dimensional spectrogram format and cannot be applied to models pretrained on speech waveforms. To address this, we propose RepAugment, an input-agnostic representation-level augmentation technique that outperforms SpecAugment, but is also suitable for respiratory sound classification with waveform pretrained models. Experimental results show that our approach outperforms the SpecAugment, demonstrating a substantial improvement in the accuracy of minority disease classes, reaching up to 7.14%.
Exploring prompts to elicit memorization in masked language model-based named entity recognition
Venue: arXiv cs.CL 2024
Year: 2024

Training data memorization in language models impacts model capability (generalization) and safety (privacy risk). This paper focuses on analyzing prompts' impact on detecting the memorization of 6 masked language model-based named entity recognition models. Specifically, we employ a diverse set of 400 automatically generated prompts, and a pairwise dataset where each pair consists of one person's name from the training set and another name out of the set. A prompt completed with a person's name serves as input for getting the model's confidence in predicting this name. Finally, the prompt performance of detecting model memorization is quantified by the percentage of name pairs for which the model has higher confidence for the name from the training set. We show that the performance of different prompts varies by as much as 16 percentage points on the same model, and prompt engineering further increases the gap. Moreover, our experiments demonstrate that prompt performance is model-dependent but does generalize across different name sets. A comprehensive analysis indicates how prompt performance is influenced by prompt properties, contained tokens, and the model's self-attention weights on the prompt.
AC-MAMBASEG: An adaptive convolution and Mamba-based architecture for enhanced skin lesion segmentation
Venue: arXiv cs.CV 2024
Year: 2024

Skin lesion segmentation is a critical task in computer-aided diagnosis systems for dermatological diseases. Accurate segmentation of skin lesions from medical images is essential for early detection, diagnosis, and treatment planning. In this paper, we propose a new model for skin lesion segmentation namely AC-MambaSeg, an enhanced model that has the hybrid CNN-Mamba backbone, and integrates advanced components such as Convolutional Block Attention Module (CBAM), Attention Gate, and Selective Kernel Bottleneck. AC-MambaSeg leverages the Vision Mamba framework for efficient feature extraction, while CBAM and Selective Kernel Bottleneck enhance its ability to focus on informative regions and suppress background noise. We evaluate the performance of AC-MambaSeg on diverse datasets of skin lesion images including ISIC-2018 and PH2; then compare it against existing segmentation methods. Our model shows promising potential for improving computer-aided diagnosis systems and facilitating early detection and treatment of dermatological diseases. Our source code will be made available at: https://github.com/vietthanh2710/AC-MambaSeg.
Fault-tolerant Consensus in Anonymous Dynamic Network
Venue: arXiv cs.DC 2024
Year: 2024

This paper studies the feasibility of reaching consensus in an anonymous dynamic network. In our model, $n$ anonymous nodes proceed in synchronous rounds. We adopt a hybrid fault model in which up to $f$ nodes may suffer crash or Byzantine faults, and the dynamic message adversary chooses a communication graph for each round. We introduce a stability property of the dynamic network -- $(T,D)$-dynaDegree for $T \geq 1$ and $n-1 \geq D \geq 1$ -- which requires that for every $T$ consecutive rounds, any fault-free node must have incoming directed links from at least $D$ distinct neighbors. These links might occur in different rounds during a $T$-round interval. $(1,n-1)$-dynaDegree means that the graph is a complete graph in every round. $(1,1)$-dynaDegree means that each node has at least one incoming neighbor in every round, but the set of incoming neighbor(s) at each node may change arbitrarily between rounds. We show that exact consensus is impossible even with $(1,n-2)$-dynaDegree. For an arbitrary $T$, we show that for crash-tolerant approximate consensus, $(T,\lfloor n/2 \rfloor)$-dynaDegree and $n > 2f$ are together necessary and sufficient, whereas for Byzantine approximate consensus, $(T,\lfloor (n+3f)/2 \rfloor)$-dynaDegree and $n > 5f$ are together necessary and sufficient.
TSP Escapes the $O(2^n n^2)$ Curse
Venue: arXiv cs.DS 2024
Year: 2024

The dynamic programming solution to the traveling salesman problem due to Bellman, and independently Held and Karp, runs in time $O(2^n n^2)$, with no improvement in the last sixty years. We break this barrier for the first time by designing an algorithm that runs in deterministic time $2^n n^2 / 2^{Ω(\sqrt{\log n})}$. We achieve this by strategically remodeling the dynamic programming recursion as a min-plus matrix product, for which faster-than-naïve algorithms exist.
FlexKalmanNet: A Modular AI-Enhanced Kalman Filter Framework Applied to Spacecraft Motion Estimation
Venue: arXiv cs.RO 2024
Year: 2024

The estimation of relative motion between spacecraft increasingly relies on feature-matching computer vision, which feeds data into a recursive filtering algorithm. Kalman filters, although efficient in noise compensation, demand extensive tuning of system and noise models. This paper introduces FlexKalmanNet, a novel modular framework that bridges this gap by integrating a deep fully connected neural network with Kalman filter-based motion estimation algorithms. FlexKalmanNet's core innovation is its ability to learn any Kalman filter parameter directly from measurement data, coupled with the flexibility to utilize various Kalman filter variants. This is achieved through a notable design decision to outsource the sequential computation from the neural network to the Kalman filter variant, enabling a purely feedforward neural network architecture. This architecture, proficient at handling complex, nonlinear features without the dependency on recurrent network modules, captures global data patterns more effectively. Empirical evaluation using data from NASA's Astrobee simulation environment focuses on learning unknown parameters of an Extended Kalman filter for spacecraft pose and twist estimation. The results demonstrate FlexKalmanNet's rapid training convergence, high accuracy, and superior performance against manually tuned Extended Kalman filters.
On the use of dynamical systems in cryptography
Venue: arXiv cs.CR 2024
Year: 2024

Ever since the link between nonlinear science and cryptography became apparent, the problem of applying chaotic dynamics to the construction of cryptographic systems has gained a broad audience and has been the subject of thousands of papers. Yet, the field has not found its place in mainstream cryptography, largely due to persistent weaknesses in the presented systems. The goal of this paper is to help remedy this problem in two ways. The first is by providing a new algorithm that can be used to attack -- and hence test the security of -- stream ciphers based on the iteration of a chaotic map of the interval. The second is to cast discrete dynamical systems problems in a modern cryptographic and complexity theoretic language, so that researchers working in chaos-based cryptography can begin designing cryptographic protocols that have a better chance of meeting the extreme standards of modern cryptography.
Swipe2Pair: Secure and Fast In-Band Wireless Device Pairing
Venue: arXiv cs.CR 2024
Year: 2024

Wireless device pairing is a critical security mechanism to bootstrap the secure communication between two devices without a pre-shared secret. It has been widely used in many Internet of Things (IoT) applications, such as smart-home and smart-health. Most existing device pairing mechanisms are based on out-of-band channels, e.g., extra sensors or hardware, to validate the proximity of pairing devices. However, out-of-band channels are not universal across all wireless devices, so such a scheme is limited to certain application scenarios or conditions. On the other hand, in-band channel-based device pairing seeks universal applicability by only relying on wireless interfaces. Existing in-band channel-based pairing schemes either require multiple antennas separated by a good distance on one pairing device, which is not feasible in certain scenarios, or require users to repeat multiple sweeps, which is not optimal in terms of usability. Therefore, an in-band wireless device pairing scheme providing high security while maintaining high usability (simple pairing process and minimal user intervention) is highly desired. In this work, we propose an easy-to-use mutual authentication device pairing scheme, named Swipe2Pair, based on the proximity of pairing devices and randomization of wireless transmission power. We conduct extensive security analysis and collect considerable experimental data under various settings across different environments. Experimental results show that Swipe2Pair achieves high security and usability. It only takes less than one second to complete the pairing process with a simple swipe of one device in front of the other.
Verifying SQL Queries using Theories of Tables and Relations
Venue: arXiv cs.LO 2024
Year: 2024

We present a number of first- and second-order extensions to SMT theories specifically aimed at representing and analyzing SQL queries with join, projection, and selection operations. We support reasoning about SQL queries with either bag or set semantics for database tables. We provide the former via an extension of a theory of finite bags and the latter via an extension of the theory of finite relations. Furthermore, we add the ability to reason about tables with null values by introducing a theory of nullable sorts based on an extension of the theory of algebraic datatypes. We implemented solvers for these theories in the SMT solver cvc5 and evaluated them on a set of benchmarks derived from public sets of SQL equivalence problems.
Analyzing Emotional Trends from X platform using SenticNet: A Comparative Analysis with Cryptocurrency Price
Venue: arXiv cs.CL 2024
Year: 2024

This study delves into the relationship between emotional trends from X platform data and the market dynamics of well-known cryptocurrencies Cardano, Binance, Fantom, Matic, and Ripple over the period from October 2022 to March 2023. Leveraging SenticNet, we identified emotions like Fear and Anxiety, Rage and Anger, Grief and Sadness, Delight and Pleasantness, Enthusiasm and Eagerness, and Delight and Joy. Following data extraction, we segmented each month into bi-weekly intervals, replicating this process for price data obtained from Finance-Yahoo. Consequently, a comparative analysis was conducted, establishing connections between emotional trends observed across bi-weekly intervals and cryptocurrency prices, uncovering significant correlations between emotional sentiments and coin valuations.
Structure-Preserving Network Compression Via Low-Rank Induced Training Through Linear Layers Composition
Venue: arXiv cs.LG 2024
Year: 2024

Deep Neural Networks (DNNs) have achieved remarkable success in addressing many previously unsolvable tasks. However, the storage and computational requirements associated with DNNs pose a challenge for deploying these trained models on resource-limited devices. Therefore, a plethora of compression and pruning techniques have been proposed in recent years. Low-rank decomposition techniques are among the approaches most utilized to address this problem. Compared to post-training compression, compression-promoted training is still under-explored. In this paper, we present a theoretically-justified technique termed Low-Rank Induced Training (LoRITa), that promotes low-rankness through the composition of linear layers and compresses by using singular value truncation. This is achieved without the need to change the structure at inference time or require constrained and/or additional optimization, other than the standard weight decay regularization. Moreover, LoRITa eliminates the need to (i) initialize with pre-trained models, (ii) specify rank selection prior to training, and (iii) compute SVD in each iteration. Our experimental results (i) demonstrate the effectiveness of our approach using MNIST on Fully Connected Networks, CIFAR10 on Vision Transformers, and CIFAR10/100 and ImageNet on Convolutional Neural Networks, and (ii) illustrate that we achieve either competitive or state-of-the-art results when compared to leading structured pruning and low-rank training methods in terms of FLOPs and parameters drop. Our code is available at \url{https://github.com/XitongSystem/LoRITa/tree/main}.
SketchGPT: Autoregressive Modeling for Sketch Generation and Recognition
Venue: arXiv cs.CV 2024
Year: 2024

We present SketchGPT, a flexible framework that employs a sequence-to-sequence autoregressive model for sketch generation, and completion, and an interpretation case study for sketch recognition. By mapping complex sketches into simplified sequences of abstract primitives, our approach significantly streamlines the input for autoregressive modeling. SketchGPT leverages the next token prediction objective strategy to understand sketch patterns, facilitating the creation and completion of drawings and also categorizing them accurately. This proposed sketch representation strategy aids in overcoming existing challenges of autoregressive modeling for continuous stroke data, enabling smoother model training and competitive performance. Our findings exhibit SketchGPT's capability to generate a diverse variety of drawings by adding both qualitative and quantitative comparisons with existing state-of-the-art, along with a comprehensive human evaluation study. The code and pretrained models will be released on our official GitHub.