Reviewerly/exHarmony huggingface dataset:
Visual Analytics for Phishing Scam Identification in Blockchain Transactions with Multiple Model Comparison

The phishing scam is a major kind of fraudulence in blockchain. And it has become an urgent issue to discern and prevent the fraudulent behaviors. However, the large-scale and dynamic nature of transaction network imposes great challenges on the identification and analysis. While there have been many sophisticated machine learning approaches providing predictive capability in terms of detecting such cases, they usually offer little insight into the essence of those behaviors and the occasion when phishing scam activities happen. Motivated by these shortcomings and bottlenecks, this paper proposes a suite of visual analytical methods for interpretable and explorable fraudulence identification in large-scale blockchain transaction networks, incorporating an anomaly detection model based on multiple feature extraction manners. In this paper, we adopt two types of graph embedding methods and variable derivation to generate features from transaction data. Then we use machine learning classification approaches to fit the three sets of features. Evaluations show that all kinds of features perform well in classification. Besides, we design an interactive visualization system displaying the transaction networks and classification models, which allows users better explore the data and understand the models. Furthermore, we demonstrate two cases through the visualization system to unearth fraudulent patterns and interpret classification results. Finally, we close with discussions for further improvements of our models and system.
Graph-based Latent Space Traversal for New Molecules Discovery

Generative models provide an efficient way to analyze and understand unlabeled data, creating the latent space for data modeling and generation. Since the interpretation of latent space usually requires implicit expert knowledge, this human-centered feature makes visual analytic methods effective. In the filed of computer chemistry, some research have applied generative models to generate chemical spaces and generated new molecules by sampling in the latent space. However, the latent space is typically high and sparse, and there may be a large number of “dead zones”, which may lead to decoding sample points from the latent space are noisy or invalid. Therefore, it is extremely challenging to efficiently search and traverse the latent space and generate new molecules with the desired properties. This paper aims to propose a visualization system for interactive exploration of latent space, which inspires the researchers to design new potential molecules with desired properties. The main work of this paper is as follows: First, we investigate a series of literature on the application of generative networks to drug design and synthesis, and interview experts with computer chemistry background to summarize the requirements and tasks. Second, based on the above requirements and tasks, we propose a graph-based latent space traversal and interpolation algorithms and neighborhood sampling algorithms. This can improve the number of generated potential molecules and the speed of discovery of similar molecules. Then, we conduct comparison experiments to verify the effectiveness of the algorithms. Finally, we design visualization system and then conduct the case study and user study to verify the effectiveness of the visualization system.
Naturality: A Natural Reflection of Chinese Calligraphy

We present a machine learning-based interactive video installation powered by CLIP and diffusion models and inspired by the concept of naturality in traditional Chinese calligraphy. The artwork explores contemporary interpretations of this traditional concept through practical methods in Artificial Intelligence Generated Content (AIGC). Technically, the algorithms are based on state-of-the-art perceptual and generative models, incorporating multi-dimensional controls over text-to-image and image-to-image translation; conceptually, this real-time art installation extends the discussion brought by Xu Bing’s pieces Book from the Sky and Square Word Calligraphy. The project explores the possibility of AIGC in bridging human creativity and natural randomness, as well as a shifting creative paradigm enhanced by AI knowledge, perception, and association.
Dreaming Phantom in Immersive Experience: AIGC For Artistic Practice

Artificial Intelligent Generation Content (AIGC), has been widely disseminated in the fields of technology, academia, and the arts. This project explores the application of various AI tools and the visualization of dream experiences through multimedia. It utilizes AI-generated multimodal materials as perceptible dream content, employs light and mechanical installations to create immersive dream atmospheres, and employs a fictional AI-Mulan narrator to recount her dream story. Through artistic practice, it delves into Mulan's unconscious realm and conducts a psychoanalysis of a historical figure. It represents an interdisciplinary exploration of art and psychoanalysis through AI visualization.
Urban Symphony: An AI and Data-Driven Approach to Real-Time Animation for Public Digital Art

Projection mapping is a form of urban public art that uses light and video to transform buildings and structures into dynamic canvases. However, producing high-quality projection mapping content with compelling storytelling requires extensive time and resources, as it involves integrating local culture, urban spatial understanding, and animation production. To address this challenge, this paper proposes a method that combines artistic co-creation with AI, audio-visualization, and data-visualization techniques. The authors present a case study: “Urban Symphony,” an immersive public art installation that showcases our method and leverages AI and data-driven storytelling. This method fosters interdisciplinary research collaboration and explores the potential of projection mapping as a bridge between art, technology, and society. The paper describe the motivation, design, and production of the artwork, the outcomes of the performance, and the challenges and limitations of our method. The authors also suggest future directions for further improvement and exploration in this domain.
A Visual Analytic System for Ranking Multi-attribute Data Using Multi-level Pareto Frontier

Ranking data items is a core step of decision making in many scenarios, such as investigating companies in finance and evaluating players in sports. While sorting by a single attribute is often trivial, ranking based on multi-attribute is often fuzzy, meaning that the goals, constraints, and weights of factors are not well-defined. Existing techniques use aggregation or dimension reduction to map the data along a single axis, which destroys the high-dimensional structure of data. In this paper, we aim to reduce the scope of data instead of the dimensionality in ranking tasks. In this way, users make decisions among a few data items based on complete information, instead of ranking many items based on inaccurate information. We introduce the concept of Pareto frontier for partitioning the data into multiple groups. A visual analytic system with two coordinated views is designed for users to rank data in an individual group or compare items in multiple groups. We evaluate the effectiveness of the proposed system through case studies and usability through a user study.
Simonstown: An AI-facilitated Interactive Story of Love, Life, and Pandemic

We present an interactive story named Simonstown that demonstrates the love and life of ordinary people in the fictional setting of a fatal pandemic. Technically, the artwork integrates different Artificial Intelligence (AI) technologies in the whole production pipeline, including concept formation, creation, and presentation stages; artistically, this interactive film explores the relationship between human and environment in the contemporary context, especially infused with advanced technologies in daily life. The project serves as a demonstration and case study of AI-facilitated interactive storytelling, including better control with AI and how they integrate with live image projects, as well as using the stand-alone camera for real-time synchronization. Our results highlight the significant contribution of AI in visualizing intricate story branching, translation, and adaptation, presenting AI visualization as a distinct, specialized, and well-suited tool for interactive filmmaking.
FitYou: A Personalized Dashboard for Health Data

We introduce FitYou, an interactive dashboard for health data visualizations from wearable devices. It supports the exploration of health trends and health attribute correlations. Wearable devices are small enough to wear on the body and that can be equipped with sensors to track various physical and physiological attributes like heart rate, steps taken, and sleep quality, to just mention a few. By exploiting those attribute values we can visually monitor health-related issues, for example during sports activities like training, a sports competition, or a rehabilitation phase. Activity levels are hard to detect without such support but can help to improve someone’s overall well-being. We illustrate the usefulness of FitYou by means of some application examples. Finally, we discuss the challenges and limitations of the dashboard by focusing on algorithmic, visual, and perceptual issues.
Designing Personalized Pedagogical AI Agents to Support Children's Exploratory Learning

Can a social robot support children to become better learners? How personalizing a social robot's behavior can encourage learner exploration.
Performance Characterization of Large Language Models on High-Speed Interconnects

Large Language Models (LLMs) have recently gained significant popularity due to their ability to generate human-like text and perform a wide range of natural language processing tasks. Training these models usually requires a large amount of computational resources and is often done in a distributed manner. The use of high-speed interconnects can significantly influence the efficiency of distributed training. Therefore, there poses a need for systematic studies to explore the distributed training characteristics of these models on high-speed interconnects. This paper presents a comprehensive performance characterization of representative large language models: GPT, BERT, and T5. We evaluate their training performance in terms of iteration time, interconnect utilization, and scalability, over different high-speed interconnects and communication protocols, including TCP/IP, IPoIB, and RDMA. We observe that interconnects play a vital role in LLM training. Specifically, RDMA-100 Gbps outperforms IPoIB-100 Gbps and TCP/IP-10 Gbps by an average of 2.51x and 4.79x regarding training iteration time, and scores the highest interconnect utilization (up to 60 Gbps) in both strong and weak scaling, compared to IPoIB with up to 20 Gbps and TCP/IP with up to 9 Gbps, leading to the shortest training time. We also observe that larger models tend to have higher requirements for communication bandwidth, especially for AllReduce during backward propagation, which can take up to 91.12% of training time. Through our evaluation, we envision opportunities to improve the communication time for better training performance of LLMs. We extensively explore and summarize the role communication plays in distributed LLM training.
Visual Question Answering in Remote Sensing with Cross-Attention and Multimodal Information Bottleneck

In this research, we deal with the problem of visual question answering (VQA) in remote sensing. While remotely sensed images contain information significant for the task of identification and object detection, they pose a great challenge in their processing because of high dimensionality, volume and redundancy. Furthermore, processing image information jointly with language features adds additional constraints, such as mapping the corresponding image and language features. To handle this problem, we propose a cross attention based approach combined with information maximization. The CNN-LSTM based cross-attention highlights the information in the image and language modalities and establishes a connection between the two, while information maximization learns a low dimensional bottleneck layer, that has all the relevant information required to carry out the VQA task. We evaluate our method on two VQA remote sensing datasets of different resolutions. For the high resolution dataset, we achieve an overall accuracy of 79.11% and 73.87% for the two test sets while for the low resolution dataset, we achieve an overall accuracy of 85.98%.
Deform-FPN: A Novel FPN with Deformable Convolution for Multi-Scale SAR Ship Detection

Ship detection from Synthetic Aperture Radar (SAR) images is of great importance. However, the diversity of ship target scales increases the difficulty of detection. To solve this problem, we propose a novel FPN which is enhanced by de-formable convo-lution, called Deform-FPN. Deformable convolution realizes multi-scale adaptive geometric deformation modeling of ships, and can extract multi-scale features of ships with strong ex-pression ability. The multi-level deformable convolution layers enhance the feature extraction and feature fusion capabilities. Specifically, we add deformable convolution to the backbone and lateral connection of Deform-FPN to improve the feature extraction ability. Experimental results on the SAR ship detec-tion dataset (SSDD) reveal the state-of-the-art performance of Deform-FPN, in contrast to other methods based on convolu-tional neural network (CNN). The experimental results show that Deform-FPN offers a 56.5% mAP that is superior to the suboptimal model DCN by 1.5%. In addition, we conducted ablation experiments to verify the effectiveness of the structure of the Deform-FPN we proposed.
Quantum-Assisted Variational Segmentation for Image-to-Image Wildfire Detection Using Satellite Data

The quantum computing community has been searching for suitable applications to demonstrate the potential of near-term quantum devices. Quantum machine learning is a potential candidate, particularly using models that cannot be efficiently simulated with classical computers [1] , [2] . This work focuses on a transition phase of quantum computers where the quantum machine learning model is still simulable classically but projected not to be simulable as the size of the model grows. Ultimately quantum computers may have advantages for high-dimensional real-world problems. Due to the limited number of qubits in current noisy intermediate-scale quantum (NISQ) devices, the direct application of quantum computers in high dimensional data is not feasible. To remedy this problem, an encoder-decoder architecture can be utilized. The encoder model would transform the high-dimensional data into a compact representation, to a level that small quantum computers can be used today (or in the near future), and the decoder would take the quantum processed outputs back to the high-dimensional space [3] .
Proba-V Multi-Temporal Super-Resolution Guided by Sentinel-2

Multi-image super-resolution (MISR) is a technique used to increase the spatial resolution of images acquired by remote sensing platforms by combining the images acquired through multiple revisits. Supervised training of MISR models requires collecting high-resolution images to be used as ground truth. Except for a few special cases, this involves acquiring images from a different satellite, resulting in a shift in the optical and radiometric characteristics with respect to the sensor to be super-resolved. In this paper, we explore the use of Sentinel-2 images to train a MISR model for Proba-V images and highlight the challenges of this pursuit.
Developments of L2B Soil and Mineral Products in the Frame of the Development of the CHIME-E2E Simulator

The Copernicus Hyperspectral Imaging Mission for the Environment (CHIME) is a new ESA Earth Observation mission which consists in developing a hyperspectral satellite to support EU policies on the management of natural resources, ultimately helping to address the global issue of food security. One of the mission activities is associated to the development of the CHIME-E2E (End-to-End) Performance Simulator that shall be used to evaluate the sensor design and future processing modules provided by the partners by simulating future CHIME images and thematic products. In the frame of this activity, the CHIME Mission Advisory Group (MAG) has identified a collection of five core high priority products (HPP) that includes the retrieval of canopy nitrogen, leaf nitrogen content, leaf mass/area, soil organic carbon content (SOC) and kaolinite abundance. In this paper, we present the first results of applying the L2B prototype processing to hyperspectral airborne and spatial imagery used to simulate realistic CHIME data, to derive soil and mineral maps. The obtained results demonstrate the potential of the next generation of Copernicus missions with high spectral resolution and wide swath imaging satellite for geoscience research and applications.
Uncertainty-Aware Dynamic Learning for Cross-Domain Few-Shot Scene Classification from Remote Sensing Imagery

Cross-domain few-shot scene classification (CDFSSC) is devoted to transferring knowledge from the source domain to the target domain and facilitating few-shot classification for the target domain. However, due to the domain shifts between source and target domains, high uncertainty would be generated in the knowledge transfer process, leading to unreliable cross-domain learning, which degenerates classification performance on the target domain severely. Thus, in this paper, aiming to reduce the interference of high uncertainty and improve the reliability of cross-domain knowledge transfer, a novel uncertainty-aware dynamic learning (UDL) framework is proposed for CDFSSC from remote sensing imagery. First, a mean-teacher architecture combining pseudo-labeling and consistency regularization is utilized to achieve cross-domain learning. Second, a UDL strategy is proposed to divide data into positive and negative samples based on a well-designed uncertainty-aware dynamic threshold, conducting positive and negative learning respectively, to advance a more reliable knowledge transfer. Third, to further improve cross-domain capability, a self-entropy loss is designed to reduce the epistemic uncertainty of the model. Extensive experiment results indicate the superiority of our proposed methods.
Feature Adversarial Network for Multimodal Template Matching

Recently generative adversarial network (GAN) has been explored to multimodal template matching. Existing GAN-based multimodal template matching methods exploit the image generation to transform the multimodal template matching task as the unimodal one. However, image synthesis-based multimodal template matching methods relay on the generated image quality, which is unstable. Towards this end, this paper proposes a feature adversarial network, which maps different modal images into a common subspace and learns the correlation in the subspace. Specifically, the feature mapper is designed to map the multimodal features into intermediate features, and the modality discriminator is proposed to optimize the multimodal intermediate features until they are indistinguishable. Thus, an effective common feature subspace is generated for correlation learning. The experimental results on a public dataset demonstrate the superiority of the proposed method.
Characterization of Slow Slip Events from Gnss Data with Deep Learning

Detecting and characterizing slow slip events (SSEs) in Global Navigation Satellite System (GNSS) time series is challenging and multi-station deep-learning approaches are still little explored. The main difficulty is the high level of noise in GNSS time series. The noise affecting GNSS measurements is spatially and temporally correlated, which requires setting up multi-station methods to better characterize the spatial extent of slow slip events. Here, we develop and compare different deep-learning approaches to detect and characterize SSEs in GNSS data, showing that methods embedding the spatial information outperform time-series-based approaches, with spatiotemporal models being the most promising and flexible on real GNSS data.
Unsupervised Domain Adaptation Using Generative Adversarial Network for Extreme Events Monitoring

In a traditional Machine Learning setting, it is assumed that the training and test dataset belong to the same feature space and the same distribution. For practical real-world scenarios, this is not always true. The model which is trained on the source dataset might give poor results on the target dataset (during testing/inferencing). We address this challenge using Unsupervised Domain Adaptation which deals with the situation in which the network is trained on the labeled source domain data and unlabeled data belonging to a different but related target domain and it gives improved results when tested on the target domain data. Also, generating data considering two different but related domains into account is a challenge. In this work, we address this challenge by proposing a framework for developing a cross-domain Generative Adversarial Network (GAN) with the aim of achieving unsupervised domain adaptation which applies to LiDAR data appertaining to extreme events. The dataset used for the study is the pre- and post-earthquake LiDAR (Light Detection and Ranging) dataset belonging to the Kumamoto earthquake of 2016. A comprehensive comparative analysis is conducted in this study to examine the efficacy of urban damage classification, both with and without the integration of unsupervised domain adaptation. We envisage that our framework can be used for performing urban damage classification in an efficient manner with increased accuracy.
Trident Cooperation Network for Building Extraction and Height Estimation

Building extraction and height estimation provide solid fundamentals for reconstructing city morphologies and investigating urban planning. To this aim, the DFC23 establishes a large-scale and multi-modal benchmark for multi-task learning of building reconstruction. However, the problems of data limitation and fore-background confusion severely inhibit the performance of the model. In this work, we propose a novel trident cooperation network (TCNet) to perform end-to-end building extraction and height estimation using RGB and SAR data. Specifically, to enrich the feature representation and generalization of the shared backbone, we introduce a vision transformer adapter to inject vision-specific inductive biases and design a cross-modal fusion (CMF) module to effectively aggregate features from multi-modal data. For downstream visual tasks, we construct trident decoders including a detector, a lightweight MLP segmentation head, and a pixel-wise regression head. Moreover, to highlight the foreground object, we use the binary mask predicted by the MLP head to cooperate with the height estimation map predicted by the estimator. And the weighted sub-task losses are gathered to optimize our TCNet. Experimental results show the effectiveness of our method, ranking 2nd in the test phase of the contest.