OpenRaiser/Intern-Atlas huggingface dataset:
Progressive Alignment with VLM-LLM Feature to Augment Defect Classification for the ASE Dataset
Venue: arXiv cs.CV 2024
Year: 2024

Traditional defect classification approaches are facing with two barriers. (1) Insufficient training data and unstable data quality. Collecting sufficient defective sample is expensive and time-costing, consequently leading to dataset variance. It introduces the difficulty on recognition and learning. (2) Over-dependence on visual modality. When the image pattern and texture is monotonic for all defect classes in a given dataset, the performance of conventional AOI system cannot be guaranteed. In scenarios where image quality is compromised due to mechanical failures or when defect information is inherently difficult to discern, the performance of deep models cannot be guaranteed. A main question is, "how to solve those two problems when they occur at the same time?" The feasible strategy is to explore another feature within dataset and combine an eminent vision-language model (VLM) and Large-Language model (LLM) with their astonishing zero-shot capability. In this work, we propose the special ASE dataset, including rich data description recorded on image, for defect classification, but the defect feature is uneasy to learn directly. Secondly, We present the prompting for VLM-LLM against defect classification with the proposed ASE dataset to activate extra-modality feature from images to enhance performance. Then, We design the novel progressive feature alignment (PFA) block to refine image-text feature to alleviate the difficulty of alignment under few-shot scenario. Finally, the proposed Cross-modality attention fusion (CMAF) module can effectively fuse different modality feature. Experiment results have demonstrated our method's effectiveness over several defect classification methods for the ASE dataset.
Multi-level Graph Subspace Contrastive Learning for Hyperspectral Image Clustering
Venue: arXiv cs.CV 2024
Year: 2024

Hyperspectral image (HSI) clustering is a challenging task due to its high complexity. Despite subspace clustering shows impressive performance for HSI, traditional methods tend to ignore the global-local interaction in HSI data. In this study, we proposed a multi-level graph subspace contrastive learning (MLGSC) for HSI clustering. The model is divided into the following main parts. Graph convolution subspace construction: utilizing spectral and texture feautures to construct two graph convolution views. Local-global graph representation: local graph representations were obtained by step-by-step convolutions and a more representative global graph representation was obtained using an attention-based pooling strategy. Multi-level graph subspace contrastive learning: multi-level contrastive learning was conducted to obtain local-global joint graph representations, to improve the consistency of the positive samples between views, and to obtain more robust graph embeddings. Specifically, graph-level contrastive learning is used to better learn global representations of HSI data. Node-level intra-view and inter-view contrastive learning is designed to learn joint representations of local regions of HSI. The proposed model is evaluated on four popular HSI datasets: Indian Pines, Pavia University, Houston, and Xu Zhou. The overall accuracies are 97.75%, 99.96%, 92.28%, and 95.73%, which significantly outperforms the current state-of-the-art clustering methods.
DiffCJK: Conditional Diffusion Model for High-Quality and Wide-coverage CJK Character Generation
Venue: arXiv cs.CV 2024
Year: 2024

Chinese, Japanese, and Korean (CJK), with a vast number of native speakers, have profound influence on society and culture. The typesetting of CJK languages carries a wide range of requirements due to the complexity of their scripts and unique literary traditions. A critical aspect of this typesetting process is that CJK fonts need to provide a set of consistent-looking glyphs for approximately one hundred thousand characters. However, creating such a font is inherently labor-intensive and expensive, which significantly hampers the development of new CJK fonts for typesetting, historical, aesthetic, or artistic purposes. To bridge this gap, we are motivated by recent advancements in diffusion-based generative models and propose a novel diffusion method for generating glyphs in a targeted style from a single conditioned, standard glyph form. Our experiments show that our method is capable of generating fonts of both printed and hand-written styles, the latter of which presents a greater challenge. Moreover, our approach shows remarkable zero-shot generalization capabilities for non-CJK but Chinese-inspired scripts. We also show our method facilitates smooth style interpolation and generates bitmap images suitable for vectorization, which is crucial in the font creation process. In summary, our proposed method opens the door to high-quality, generative model-assisted font creation for CJK characters, for both typesetting and artistic endeavors.
Fair Machine Guidance to Enhance Fair Decision Making in Biased People
Venue: arXiv cs.HC 2024
Year: 2024

Teaching unbiased decision-making is crucial for addressing biased decision-making in daily life. Although both raising awareness of personal biases and providing guidance on unbiased decision-making are essential, the latter topics remains under-researched. In this study, we developed and evaluated an AI system aimed at educating individuals on making unbiased decisions using fairness-aware machine learning. In a between-subjects experimental design, 99 participants who were prone to bias performed personal assessment tasks. They were divided into two groups: a) those who received AI guidance for fair decision-making before the task and b) those who received no such guidance but were informed of their biases. The results suggest that although several participants doubted the fairness of the AI system, fair machine guidance prompted them to reassess their views regarding fairness, reflect on their biases, and modify their decision-making criteria. Our findings provide insights into the design of AI systems for guiding fair decision-making in humans.
Stylizing Sparse-View 3D Scenes with Hierarchical Neural Representation
Venue: arXiv cs.CV 2024
Year: 2024

Recently, a surge of 3D style transfer methods has been proposed that leverage the scene reconstruction power of a pre-trained neural radiance field (NeRF). To successfully stylize a scene this way, one must first reconstruct a photo-realistic radiance field from collected images of the scene. However, when only sparse input views are available, pre-trained few-shot NeRFs often suffer from high-frequency artifacts, which are generated as a by-product of high-frequency details for improving reconstruction quality. Is it possible to generate more faithful stylized scenes from sparse inputs by directly optimizing encoding-based scene representation with target style? In this paper, we consider the stylization of sparse-view scenes in terms of disentangling content semantics and style textures. We propose a coarse-to-fine sparse-view scene stylization framework, where a novel hierarchical encoding-based neural representation is designed to generate high-quality stylized scenes directly from implicit scene representations. We also propose a new optimization strategy with content strength annealing to achieve realistic stylization and better content preservation. Extensive experiments demonstrate that our method can achieve high-quality stylization of sparse-view scenes and outperforms fine-tuning-based baselines in terms of stylization quality and efficiency.
Computational Propaganda Theory and Bot Detection System: Critical Literature Review
Venue: arXiv cs.SI 2024
Year: 2024

According to the classical definition, propaganda is the management of collective attitudes by manipulation of significant symbols. However this definition has changed to computational propaganda, the way manipulation takes place in digital medium. Computational propaganda is the use of algorithms, automation and human curation to purposefully distribute misleading information over social media networks to manipulate public opinion, for political polarization etc. Digital media platforms have introduced new modalities of propaganda such as the use of social bots and state-organized 'troll armies' for social astroturfing to simulate public support or opposition towards a particular topic. Along with this digital media has blurred the line between different forms of propaganda. Hence existing conceptual and epistemological frameworks in propaganda studies need a revision. One of the methods to detect the computational propaganda is to identify automation and bots. Many supervised machine learning based frameworks have been proposed for bot detection but these systems can only identify single accounts, not the coordinated activities of botnets and also these systems depend on the data structure provided by the social media platforms. Similarly, current systems have not included the image features in their detection system. Most of the systems are mainly built for Twitter while there are still uncharted areas of research in other social media platforms. Therefore, there are many unexplored research questions and methods in bot detection systems.
Robust Anthropomorphic Robotic Manipulation through Biomimetic Distributed Compliance
Venue: arXiv cs.RO 2024
Year: 2024

The impressive capabilities of humans to robustly perform manipulation relies on compliant interactions, enabled through the structure and materials spatially distributed in our hands. We propose by mimicking this distributed compliance in an anthropomorphic robotic hand, the open-loop manipulation robustness increases and observe the emergence of human-like behaviours. To achieve this, we introduce the ADAPT Hand equipped with tunable compliance throughout the skin, fingers, and the wrist. Through extensive automated pick-and-place tests, we show the grasping robustness closely mirrors an estimated geometric theoretical limit, while `stress-testing' the robot hand to perform 800+ grasps. Finally, 24 items with largely varying geometries are grasped in a constrained environment with a success rate of 93%. We demonstrate the hand-object self-organization behavior underlines this extreme robustness, where the hand automatically exhibits different grasp types depending on object geometries. Furthermore, the robot grasp type mimics a natural human grasp with a direct similarity of 68%.
Function spaces for orbit-finite sets
Venue: arXiv cs.LO 2024
Year: 2024

Orbit-finite sets are a generalisation of finite sets, and as such support many operations allowed for finite sets, such as pairing, quotienting, or taking subsets. However, they do not support function spaces, i.e. if X and Y are orbit-finite sets, then the space of finitely supported functions from X to Y is not orbit-finite. In this paper we propose two solutions to this problem: one is obtained by generalising the notion of orbit-finite set, and the other one is obtained by restricting it. In both cases, function spaces and the original closure properties are retained. Curiously, both solutions are "linear": the generalisation is based on linear algebra, while the restriction is based on linear logic.
Online Learning of Joint-Muscle Mapping Using Vision in Tendon-driven Musculoskeletal Humanoids
Venue: arXiv cs.RO 2024
Year: 2024

The body structures of tendon-driven musculoskeletal humanoids are complex, and accurate modeling is difficult, because they are made by imitating the body structures of human beings. For this reason, we have not been able to move them accurately like ordinary humanoids driven by actuators in each axis, and large internal muscle tension and slack of tendon wires have emerged by the model error between its geometric model and the actual robot. Therefore, we construct a joint-muscle mapping (JMM) using a neural network (NN), which expresses a nonlinear relationship between joint angles and muscle lengths, and aim to move tendon-driven musculoskeletal humanoids accurately by updating the JMM online from data of the actual robot. In this study, the JMM is updated online by using the vision of the robot so that it moves to the correct position (Vision Updater). Also, we execute another update to modify muscle antagonisms correctly (Antagonism Updater). By using these two updaters, the error between the target and actual joint angles decrease to about 40% in 5 minutes, and we show through a manipulation experiment that the tendon-driven musculoskeletal humanoid Kengoro becomes able to move as intended. This novel system can adapt to the state change and growth of robots, because it updates the JMM online successively.
Can Edge Computing fulfill the requirements of automated vehicular services using 5G network ?
Venue: arXiv cs.NI 2024
Year: 2024

Communication and computation services supporting Connected and Automated Vehicles (CAVs) are characterized by stringent requirements, in terms of response time and reliability. Fulfilling these requirements is crucial for ensuring road safety and traffic optimization. The conceptually simple solution of hosting these services in the vehicles increases their cost (mainly due to the installation and maintenance of computation infrastructure) and may drain their battery excessively. Such disadvantages can be tackled via Multi-Access Edge Computing (MEC), consisting in deploying computation capability in network nodes deployed close to the devices (vehicles in this case), such as to satisfy the stringent CAV requirements. However, it is not yet clear under which conditions MEC can support CAV requirements and for which services. To shed light on this question, we conduct a simulation campaign using well-known open-source simulation tools, namely OMNeT++, Simu5G, Veins, INET, and SUMO. We are thus able to provide a reality check on MEC for CAV, pinpointing what are the computation capacities that must be installed in the MEC, to support the different services, and the amount of vehicles that a single MEC node can support. We find that such parameters must vary a lot, depending on the service considered. This study can serve as a preliminary basis for network operators to plan future deployment of MEC to support CAV.
SARIS: Accelerating Stencil Computations on Energy-Efficient RISC-V Compute Clusters with Indirect Stream Registers
Venue: arXiv cs.MS 2024
Year: 2024

Stencil codes are performance-critical in many compute-intensive applications, but suffer from significant address calculation and irregular memory access overheads. This work presents SARIS, a general and highly flexible methodology for stencil acceleration using register-mapped indirect streams. We demonstrate SARIS for various stencil codes on an eight-core RISC-V compute cluster with indirect stream registers, achieving significant speedups of 2.72x, near-ideal FPU utilizations of 81%, and energy efficiency improvements of 1.58x over an RV32G baseline on average. Scaling out to a 256-core manycore system, we estimate an average FPU utilization of 64%, an average speedup of 2.14x, and up to 15% higher fractions of peak compute than a leading GPU code generator.
CLIPping the Limits: Finding the Sweet Spot for Relevant Images in Automated Driving Systems Perception Testing
Venue: arXiv cs.CV 2024
Year: 2024

Perception systems, especially cameras, are the eyes of automated driving systems. Ensuring that they function reliably and robustly is therefore an important building block in the automation of vehicles. There are various approaches to test the perception of automated driving systems. Ultimately, however, it always comes down to the investigation of the behavior of perception systems under specific input data. Camera images are a crucial part of the input data. Image data sets are therefore collected for the testing of automated driving systems, but it is non-trivial to find specific images in these data sets. Thanks to recent developments in neural networks, there are now methods for sorting the images in a data set according to their similarity to a prompt in natural language. In order to further automate the provision of search results, we make a contribution by automating the threshold definition in these sorted results and returning only the images relevant to the prompt as a result. Our focus is on preventing false positives and false negatives equally. It is also important that our method is robust and in the case that our assumptions are not fulfilled, we provide a fallback solution.
WebXR, A-Frame and Networked-Aframe as a Basis for an Open Metaverse: A Conceptual Architecture
Venue: arXiv cs.CV 2024
Year: 2024

This work proposes a WebXR-based cross-platform conceptual architecture, leveraging the A-Frame and Networked-Aframe frameworks, in order to facilitate the development of an open, accessible, and interoperable metaverse. By introducing the concept of spatial web app, this research contributes to the discourse on the metaverse, offering an architecture that democratizes access to virtual environments and extended reality through the web, and aligns with Tim Berners-Lee's original vision of the World Wide Web as an open platform in the digital realm.
Reflected Search Poisoning for Illicit Promotion
Venue: arXiv cs.CR 2024
Year: 2024

As an emerging black hat search engine optimization (SEO) technique, reflected search poisoning (RSP) allows a miscreant to free-ride the reputation of high-ranking websites, poisoning search engines with illicit promotion texts (IPTs) in an efficient and stealthy manner, while avoiding the burden of continuous website compromise as required by traditional promotion infections. However, little is known about the security implications of RSP, e.g., what illicit promotion campaigns are being distributed by RSP, and to what extent regular search users can be exposed to illicit promotion texts distributed by RSP. In this study, we conduct the first security study on RSP-based illicit promotion, which is made possible through an end-to-end methodology for capturing, analyzing, and infiltrating IPTs. As a result, IPTs distributed via RSP are found to be large-scale, continuously growing, and diverse in both illicit categories and natural languages. Particularly, we have identified over 11 million distinct IPTs belonging to 14 different illicit categories, with typical examples including drug trading, data theft, counterfeit goods, and hacking services. Also, the underlying RSP cases have abused tens of thousands of high-ranking websites, as well as extensively poisoning all four popular search engines we studied, especially Google Search and Bing. Furthermore, it is observed that benign search users are being exposed to IPTs at a concerning extent. To facilitate interaction with potential customers (victim search users), miscreants tend to embed various types of contacts in IPTs, especially instant messaging accounts. Further infiltration of these IPT contacts reveals that the underlying illicit campaigns are operated on a large scale. All these findings highlight the negative security implications of IPTs and RSPs, and thus call for more efforts to mitigate RSP-driven illicit promotion.
Non-linear Model Predictive Control for Multi-task GPS-free Autonomous Navigation in Vineyards
Venue: arXiv cs.RO 2024
Year: 2024

Autonomous navigation is the foundation of agricultural robots. This paper focuses on developing an advanced autonomous navigation system for a rover operating within row-based crops. A position-agnostic system is proposed to address the challenging situation when standard localization methods, like GPS, fail due to unfavorable weather or obstructed signals. This breakthrough is especially vital in densely vegetated regions, including areas covered by thick tree canopies or pergola vineyards. This work proposed a novel system that leverages a single RGB-D camera and a Non-linear Model Predictive Control strategy to navigate through entire rows, adapting to various crop spacing. The presented solution demonstrates versatility in handling diverse crop densities, environmental factors, and multiple navigation tasks to support agricultural activities at an extremely cost-effective implementation. Experimental validation in simulated and real vineyards underscores the system's robustness and competitiveness in both standard row traversal and target objects approach.
Certified PEFTSmoothing: Parameter-Efficient Fine-Tuning with Randomized Smoothing
Venue: arXiv cs.LG 2024
Year: 2024

Randomized smoothing is the primary certified robustness method for accessing the robustness of deep learning models to adversarial perturbations in the l2-norm, by adding isotropic Gaussian noise to the input image and returning the majority votes over the base classifier. Theoretically, it provides a certified norm bound, ensuring predictions of adversarial examples are stable within this bound. A notable constraint limiting widespread adoption is the necessity to retrain base models entirely from scratch to attain a robust version. This is because the base model fails to learn the noise-augmented data distribution to give an accurate vote. One intuitive way to overcome this challenge is to involve a custom-trained denoiser to eliminate the noise. However, this approach is inefficient and sub-optimal. Inspired by recent large model training procedures, we explore an alternative way named PEFTSmoothing to adapt the base model to learn the Gaussian noise-augmented data with Parameter-Efficient Fine-Tuning (PEFT) methods in both white-box and black-box settings. Extensive results demonstrate the effectiveness and efficiency of PEFTSmoothing, which allow us to certify over 98% accuracy for ViT on CIFAR-10, 20% higher than SoTA denoised smoothing, and over 61% accuracy on ImageNet which is 30% higher than CNN-based denoiser and comparable to the Diffusion-based denoiser.
A parameter-free clustering algorithm for missing datasets
Venue: arXiv cs.LG 2024
Year: 2024

Missing datasets, in which some objects have missing values in certain dimensions, are prevalent in the Real-world. Existing clustering algorithms for missing datasets first impute the missing values and then perform clustering. However, both the imputation and clustering processes require input parameters. Too many input parameters inevitably increase the difficulty of obtaining accurate clustering results. Although some studies have shown that decision graphs can replace the input parameters of clustering algorithms, current decision graphs require equivalent dimensions among objects and are therefore not suitable for missing datasets. To this end, we propose a Single-Dimensional Clustering algorithm, i.e., SDC. SDC, which removes the imputation process and adapts the decision graph to the missing datasets by splitting dimension and partition intersection fusion, can obtain valid clustering results on the missing datasets without input parameters. Experiments demonstrate that, across three evaluation metrics, SDC outperforms baseline algorithms by at least 13.7%(NMI), 23.8%(ARI), and 8.1%(Purity).