OpenRaiser/Intern-Atlas huggingface dataset:
Perceptual Crack Detection for Rendered 3D Textured Meshes
Venue: arXiv cs.CV 2024
Year: 2024

Recent years have witnessed many advancements in the applications of 3D textured meshes. As the demand continues to rise, evaluating the perceptual quality of this new type of media content becomes crucial for quality assurance and optimization purposes. Different from traditional image quality assessment, crack is an annoying artifact specific to rendered 3D meshes that severely affects their perceptual quality. In this work, we make one of the first attempts to propose a novel Perceptual Crack Detection (PCD) method for detecting and localizing crack artifacts in rendered meshes. Specifically, motivated by the characteristics of the human visual system (HVS), we adopt contrast and Laplacian measurement modules to characterize crack artifacts and differentiate them from other undesired artifacts. Extensive experiments on large-scale public datasets of 3D textured meshes demonstrate effectiveness and efficiency of the proposed PCD method in correct localization and detection of crack artifacts. %Specifically, We propose a full-reference crack artifact localization method that operates on a pair of input snapshots of distorted and reference 3D objects to generate a final crack map. Moreover, to quantify the performance of the proposed detection method and validate its effectiveness, we propose a simple yet effective weighting mechanism to incorporate the resulting crack map into classical quality assessment (QA) models, which creates significant performance improvement in predicting the perceptual image quality when tested on public datasets of static 3D textured meshes. A software release of the proposed method is publicly available at: https://github.com/arshafiee/crack-detection-VVM
Simulation of the abstract Tile Assembly Model Using Crisscross Slats
Venue: arXiv cs.CG 2024
Year: 2024

Tile assembly systems in the abstract Tile Assembly Model (aTAM) are computationally universal and capable of building complex shapes, but DNA-based implementations encounter formidable error rates that stifle this theoretical potential. Slat-based self-assembly is a recent development wherein DNA forms long slats that combine together in 2 layers, rather than the aTAM's square tiles in a plane. While tiles tend to bind to 2 neighboring tiles at a time, slats may bind to dozens of other slats. Large slat-based DNA constructions have been implemented in the lab with incredible resilience to many of the errors that plague tile-based constructions, but these come at a cost as slat-based systems are often more difficult to design and simulate. Also, it has not been clear if slats, with their larger sizes and different geometries, have the same theoretical capabilities as tiles. Here we show that slats do, at least at scale. We give constructions showing that any aTAM system may be simulated by a system of slats and that these can be made more efficiently, using shorter slats and a smaller scale factor, when simulating simpler classes of systems. We consider 5 classes of aTAM systems with increasing complexity, from zig-zag systems to the full class of all aTAM systems, and show how they can be converted to equivalent slat systems. Zig-zag systems can be simulated by slats at only a $2c \times 2c$ scale (where $c$ is the freely chosen cooperativity of the slats), the full class of aTAM systems at only a $5c \times 5c$ scale, and intermediate classes using scales between these. Together, these results prove that slats have the full theoretical power of aTAM tiles while providing constructions compact enough to potentially provide designs for DNA-based implementations of slat systems that are both capable of powerful algorithmic self-assembly and possessing the strong error resilience of slats.
MaskMatch: Boosting Semi-Supervised Learning Through Mask Autoencoder-Driven Feature Learning
Venue: arXiv cs.CV 2024
Year: 2024

Conventional methods in semi-supervised learning (SSL) often face challenges related to limited data utilization, mainly due to their reliance on threshold-based techniques for selecting high-confidence unlabeled data during training. Various efforts (e.g., FreeMatch) have been made to enhance data utilization by tweaking the thresholds, yet none have managed to use 100% of the available data. To overcome this limitation and improve SSL performance, we introduce \algo, a novel algorithm that fully utilizes unlabeled data to boost semi-supervised learning. \algo integrates a self-supervised learning strategy, i.e., Masked Autoencoder (MAE), that uses all available data to enforce the visual representation learning. This enables the SSL algorithm to leverage all available data, including samples typically filtered out by traditional methods. In addition, we propose a synthetic data training approach to further increase data utilization and improve generalization. These innovations lead \algo to achieve state-of-the-art results on challenging datasets. For instance, on CIFAR-100 with 2 labels per class, STL-10 with 4 labels per class, and Euro-SAT with 2 labels per class, \algo achieves low error rates of 18.71%, 9.47%, and 3.07%, respectively. The code will be made publicly available.
The Fearless Journey [Draft]
Venue: arXiv cs.PL 2024
Year: 2024

Existing minimal Object-Oriented models (OO), like Featherweight Java (FJ), are valuable for modelling programs and designing new programming languages and tools. However, their utility in developing real-world programs is limited. We introduce the 'Fearless Heart', a novel object calculus preserving FJ's minimal and extensible nature while being more suited for constructing complex, real-world applications. To illustrate the extensibility of the Fearless Heart, we extend it with Reference Capabilities (RC), creating R-Fearless. It supports mutability and other side effects while retaining the reasoning advantages of functional programming and gaining support for features that are well-known to be enabled by RC, like automatic parallelism, caching and invariants. R-Fearless is still minimal enough to allow further extensions. It is an ideal foundation for constructing both practical systems and formal models.
TS3IM: Unveiling Structural Similarity in Time Series through Image Similarity Assessment Insights
Venue: arXiv cs.LG 2024
Year: 2024

In the realm of time series analysis, accurately measuring similarity is crucial for applications such as forecasting, anomaly detection, and clustering. However, existing metrics often fail to capture the complex, multidimensional nature of time series data, limiting their effectiveness and application. This paper introduces the Structured Similarity Index Measure for Time Series (TS3IM), a novel approach inspired by the success of the Structural Similarity Index Measure (SSIM) in image analysis, tailored to address these limitations by assessing structural similarity in time series. TS3IM evaluates multiple dimensions of similarity-trend, variability, and structural integrity-offering a more nuanced and comprehensive measure. This metric represents a significant leap forward, providing a robust tool for analyzing temporal data and offering more accurate and comprehensive sequence analysis and decision support in fields such as monitoring power consumption, analyzing traffic flow, and adversarial recognition. Our extensive experimental results also show that compared with traditional methods that rely heavily on computational correlation, TS3IM is 1.87 times more similar to Dynamic Time Warping (DTW) in evaluation results and improves by more than 50% in adversarial recognition.
Impedance vs. Power Side-channel Vulnerabilities: A Comparative Study
Venue: arXiv cs.CR 2024
Year: 2024

Physical side channels emerge from the relation between internal computation or data with observable physical parameters of a chip. Previous works mostly focus on properties related to current consumption such as power consumption. The fundamental property behind current consumption occur from the impedance of the chip. Contemporary works have stared using chip impedance as a physical side channel in extracting sensitive information from computing systems. It leverages variations in intrinsic impedance of a chip across different logic states. However, there has been a lack of comparative studies. In this study, we conduct a comparative analysis of the impedance side channel, which has been limitedly explored, and the well-established power side channel. Through experimental evaluation, we investigate the efficacy of these side channels in extracting stored advanced encryption standard (AES) cryptographic key on a memory and analyze their performance. Our findings indicate that impedance analysis demonstrates a higher potential for cryptographic key extraction compared to power side-channel analysis (SCA). Moreover, we identify scenarios where power SCA does not yield satisfactory results, whereas impedance analysis proves to be more robust and effective. This work not only underscores the significance of impedance SCA in enhancing cryptographic security but also emphasizes the necessity for a deeper understanding of its mechanisms and implications.
Comparative Analysis of Advanced Feature Matching Algorithms in Challenging High Spatial Resolution Optical Satellite Stereo Scenarios
Venue: arXiv cs.CV 2024
Year: 2024

Feature matching determines the orientation accuracy for the High Spatial Resolution (HSR) optical satellite stereos, subsequently impacting several significant applications such as 3D reconstruction and change detection. However, the matching of off-track HSR optical satellite stereos often encounters challenging conditions including wide-baseline observation, significant radiometric differences, multi-temporal changes, varying spatial resolutions, inconsistent spectral resolution, and diverse sensors. In this study, we evaluate various advanced feature matching algorithms for HSR optical satellite stereos. Utilizing a specially constructed dataset from five satellites across six challenging scenarios, HSROSS Dataset, we conduct a comparative analysis of four algorithms: the traditional SIFT, and deep-learning based methods including SuperPoint + SuperGlue, SuperPoint + LightGlue, and LoFTR. Our findings highlight overall superior performance of SuperPoint + LightGlue in balancing robustness, accuracy, distribution, and efficiency, showcasing its potential in complex HSR optical satellite scenarios.
Disttack: Graph Adversarial Attacks Toward Distributed GNN Training
Venue: arXiv cs.LG 2024
Year: 2024

Graph Neural Networks (GNNs) have emerged as potent models for graph learning. Distributing the training process across multiple computing nodes is the most promising solution to address the challenges of ever-growing real-world graphs. However, current adversarial attack methods on GNNs neglect the characteristics and applications of the distributed scenario, leading to suboptimal performance and inefficiency in attacking distributed GNN training. In this study, we introduce Disttack, the first framework of adversarial attacks for distributed GNN training that leverages the characteristics of frequent gradient updates in a distributed system. Specifically, Disttack corrupts distributed GNN training by injecting adversarial attacks into one single computing node. The attacked subgraphs are precisely perturbed to induce an abnormal gradient ascent in backpropagation, disrupting gradient synchronization between computing nodes and thus leading to a significant performance decline of the trained GNN. We evaluate Disttack on four large real-world graphs by attacking five widely adopted GNNs. Compared with the state-of-the-art attack method, experimental results demonstrate that Disttack amplifies the model accuracy degradation by 2.75$\times$ and achieves speedup by 17.33$\times$ on average while maintaining unnoticeability.
Look Once to Hear: Target Speech Hearing with Noisy Examples
Venue: arXiv cs.SD 2024
Year: 2024

In crowded settings, the human brain can focus on speech from a target speaker, given prior knowledge of how they sound. We introduce a novel intelligent hearable system that achieves this capability, enabling target speech hearing to ignore all interfering speech and noise, but the target speaker. A naive approach is to require a clean speech example to enroll the target speaker. This is however not well aligned with the hearable application domain since obtaining a clean example is challenging in real world scenarios, creating a unique user interface problem. We present the first enrollment interface where the wearer looks at the target speaker for a few seconds to capture a single, short, highly noisy, binaural example of the target speaker. This noisy example is used for enrollment and subsequent speech extraction in the presence of interfering speakers and noise. Our system achieves a signal quality improvement of 7.01 dB using less than 5 seconds of noisy enrollment audio and can process 8 ms of audio chunks in 6.24 ms on an embedded CPU. Our user studies demonstrate generalization to real-world static and mobile speakers in previously unseen indoor and outdoor multipath environments. Finally, our enrollment interface for noisy examples does not cause performance degradation compared to clean examples, while being convenient and user-friendly. Taking a step back, this paper takes an important step towards enhancing the human auditory perception with artificial intelligence. We provide code and data at: https://github.com/vb000/LookOnceToHear.
Correlation Dimension of Natural Language in a Statistical Manifold
Venue: arXiv cs.CL 2024
Year: 2024

The correlation dimension of natural language is measured by applying the Grassberger-Procaccia algorithm to high-dimensional sequences produced by a large-scale language model. This method, previously studied only in a Euclidean space, is reformulated in a statistical manifold via the Fisher-Rao distance. Language exhibits a multifractal, with global self-similarity and a universal dimension around 6.5, which is smaller than those of simple discrete random sequences and larger than that of a Barabási-Albert process. Long memory is the key to producing self-similarity. Our method is applicable to any probabilistic model of real-world discrete sequences, and we show an application to music data.
Open Access Battle Damage Detection via Pixel-Wise T-Test on Sentinel-1 Imagery
Venue: arXiv cs.CV 2024
Year: 2024

In the context of recent, highly destructive conflicts in Gaza and Ukraine, reliable estimates of building damage are essential for an informed public discourse, human rights monitoring, and humanitarian aid provision. Given the contentious nature of conflict damage assessment, these estimates must be fully reproducible, explainable, and derived from open access data. This paper introduces a new method for building damage detection-- the Pixel-Wise T-Test (PWTT)-- that satisfies these conditions. Using a combination of freely-available synthetic aperture radar imagery and statistical change detection, the PWTT generates accurate conflict damage estimates across a wide area at regular time intervals. Accuracy is assessed using an original dataset of over half a million labeled building footprints spanning 12 cities across Ukraine, Palestine, Syria, and Iraq. Despite being simple and lightweight, the algorithm achieves building-level accuracy statistics (AUC=0.88 across Ukraine, 0.81 in Gaza) rivalling state of the art methods that use deep learning and high resolution imagery. The workflow is open source and deployed entirely within the Google Earth Engine environment, allowing for the generation of interactive Battle Damage Dashboards for Ukraine and Gaza that update in near-real time, allowing the public and humanitarian practitioners to immediately get estimates of damaged buildings in a given area.
ChatGPTest: opportunities and cautionary tales of utilizing AI for questionnaire pretesting
Venue: arXiv cs.CY 2024
Year: 2024

The rapid advancements in generative artificial intelligence have opened up new avenues for enhancing various aspects of research, including the design and evaluation of survey questionnaires. However, the recent pioneering applications have not considered questionnaire pretesting. This article explores the use of GPT models as a useful tool for pretesting survey questionnaires, particularly in the early stages of survey design. Illustrated with two applications, the article suggests incorporating GPT feedback as an additional stage before human pretesting, potentially reducing successive iterations. The article also emphasizes the indispensable role of researchers' judgment in interpreting and implementing AI-generated feedback.
Improving Transferable Targeted Adversarial Attack via Normalized Logit Calibration and Truncated Feature Mixing
Venue: arXiv cs.CV 2024
Year: 2024

This paper aims to enhance the transferability of adversarial samples in targeted attacks, where attack success rates remain comparatively low. To achieve this objective, we propose two distinct techniques for improving the targeted transferability from the loss and feature aspects. First, in previous approaches, logit calibrations used in targeted attacks primarily focus on the logit margin between the targeted class and the untargeted classes among samples, neglecting the standard deviation of the logit. In contrast, we introduce a new normalized logit calibration method that jointly considers the logit margin and the standard deviation of logits. This approach effectively calibrates the logits, enhancing the targeted transferability. Second, previous studies have demonstrated that mixing the features of clean samples during optimization can significantly increase transferability. Building upon this, we further investigate a truncated feature mixing method to reduce the impact of the source training model, resulting in additional improvements. The truncated feature is determined by removing the Rank-1 feature associated with the largest singular value decomposed from the high-level convolutional layers of the clean sample. Extensive experiments conducted on the ImageNet-Compatible and CIFAR-10 datasets demonstrate the individual and mutual benefits of our proposed two components, which outperform the state-of-the-art methods by a large margin in black-box targeted attacks.
Entropic Bounds on the Average Length of Codes with a Space
Venue: arXiv cs.IT 2024
Year: 2024

We consider the problem of constructing prefix-free codes in which a designated symbol, a space, can only appear at the end of codewords. We provide a linear-time algorithm to construct almost-optimal codes with this property, meaning that their average length differs from the minimum possible by at most one. We obtain our results by uncovering a relation between our class of codes and the class of one-to-one codes. Additionally, we derive upper and lower bounds to the average length of optimal prefix-free codes with a space in terms of the source entropy.
Multi-level Personalized Federated Learning on Heterogeneous and Long-Tailed Data
Venue: arXiv cs.AI 2024
Year: 2024

Federated learning (FL) offers a privacy-centric distributed learning framework, enabling model training on individual clients and central aggregation without necessitating data exchange. Nonetheless, FL implementations often suffer from non-i.i.d. and long-tailed class distributions across mobile applications, e.g., autonomous vehicles, which leads models to overfitting as local training may converge to sub-optimal. In our study, we explore the impact of data heterogeneity on model bias and introduce an innovative personalized FL framework, Multi-level Personalized Federated Learning (MuPFL), which leverages the hierarchical architecture of FL to fully harness computational resources at various levels. This framework integrates three pivotal modules: Biased Activation Value Dropout (BAVD) to mitigate overfitting and accelerate training; Adaptive Cluster-based Model Update (ACMU) to refine local models ensuring coherent global aggregation; and Prior Knowledge-assisted Classifier Fine-tuning (PKCF) to bolster classification and personalize models in accord with skewed local data with shared knowledge. Extensive experiments on diverse real-world datasets for image classification and semantic segmentation validate that MuPFL consistently outperforms state-of-the-art baselines, even under extreme non-i.i.d. and long-tail conditions, which enhances accuracy by as much as 7.39% and accelerates training by up to 80% at most, marking significant advancements in both efficiency and effectiveness.
Pseudo-Prompt Generating in Pre-trained Vision-Language Models for Multi-Label Medical Image Classification
Venue: arXiv cs.CV 2024
Year: 2024

The task of medical image recognition is notably complicated by the presence of varied and multiple pathological indications, presenting a unique challenge in multi-label classification with unseen labels. This complexity underlines the need for computer-aided diagnosis methods employing multi-label zero-shot learning. Recent advancements in pre-trained vision-language models (VLMs) have showcased notable zero-shot classification abilities on medical images. However, these methods have limitations on leveraging extensive pre-trained knowledge from broader image datasets, and often depend on manual prompt construction by expert radiologists. By automating the process of prompt tuning, prompt learning techniques have emerged as an efficient way to adapt VLMs to downstream tasks. Yet, existing CoOp-based strategies fall short in performing class-specific prompts on unseen categories, limiting generalizability in fine-grained scenarios. To overcome these constraints, we introduce a novel prompt generation approach inspirited by text generation in natural language processing (NLP). Our method, named Pseudo-Prompt Generating (PsPG), capitalizes on the priori knowledge of multi-modal features. Featuring a RNN-based decoder, PsPG autoregressively generates class-tailored embedding vectors, i.e., pseudo-prompts. Comparative evaluations on various multi-label chest radiograph datasets affirm the superiority of our approach against leading medical vision-language and multi-label prompt learning methods. The source code is available at https://github.com/fallingnight/PsPG
A Distributionally Robust Approach to Shannon Limits using the Wasserstein Distance
Venue: arXiv cs.IT 2024
Year: 2024

We consider the rate-distortion function for lossy source compression, as well as the channel capacity for error correction, through the lens of distributional robustness. We assume that the distribution of the source or of the additive channel noise is unknown and lies within a Wasserstein-2 ambiguity set of a given radius centered around a specified nominal distribution, and we look for the worst-case asymptotically optimal coding rate over such an ambiguity set. Varying the radius of the ambiguity set allows us to interpolate between the worst-case and stochastic scenarios using probabilistic tools. Our problem setting fits into the paradigm of compound source / channel models introduced by Sakrison and Blackwell, respectively. This paper shows that if the nominal distribution is Gaussian, then so is the worst-case source / noise distribution, and the compound rate-distortion / channel capacity functions admit convex formulations with Linear Matrix Inequality (LMI) constraints. These formulations yield simple closed-form expressions in the scalar case, offering insights into the behavior of Shannon limits with the changing radius of the Wasserstein-2 ambiguity set.