OpenRaiser/Intern-Atlas huggingface dataset:
Harnessing Deep Learning and Satellite Imagery for Post-Buyout Land Cover Mapping
Venue: arXiv cs.CV 2024
Year: 2024

Environmental disasters such as floods, hurricanes, and wildfires have increasingly threatened communities worldwide, prompting various mitigation strategies. Among these, property buyouts have emerged as a prominent approach to reducing vulnerability to future disasters. This strategy involves governments purchasing at-risk properties from willing sellers and converting the land into open space, ostensibly reducing future disaster risk and impact. However, the aftermath of these buyouts, particularly concerning land-use patterns and community impacts, remains under-explored. This research aims to fill this gap by employing innovative techniques like satellite imagery analysis and deep learning to study these patterns. To achieve this goal, we employed FEMA's Hazard Mitigation Grant Program (HMGP) buyout dataset, encompassing over 41,004 addresses of these buyout properties from 1989 to 2017. Leveraging Google's Maps Static API, we gathered 40,053 satellite images corresponding to these buyout lands. Subsequently, we implemented five cutting-edge machine learning models to evaluate their performance in classifying land cover types. Notably, this task involved multi-class classification, and our model achieved an outstanding ROC-AUC score of 98.86%
Space Networking Kit: A Novel Simulation Platform for Emerging LEO Mega-constellations
Venue: arXiv cs.NI 2024
Year: 2024

This paper presents SNK, a novel simulation platform designed to evaluate the network performance of constellation systems for global Internet services. SNK offers realtime communication visualization and supports the simulation of routing between edge node of network. The platform enables the evaluation of routing and network performance metrics such as latency, stretch, network capacity, and throughput under different network structures and density. The effectiveness of SNK is demonstrated through various simulation cases, including the routing between fixed edge stations or mobile edge stations and analysis of space network structures.
InstantID: Zero-shot Identity-Preserving Generation in Seconds
Venue: arXiv cs.CV 2024
Year: 2024

There has been significant progress in personalized image synthesis with methods such as Textual Inversion, DreamBooth, and LoRA. Yet, their real-world applicability is hindered by high storage demands, lengthy fine-tuning processes, and the need for multiple reference images. Conversely, existing ID embedding-based methods, while requiring only a single forward inference, face challenges: they either necessitate extensive fine-tuning across numerous model parameters, lack compatibility with community pre-trained models, or fail to maintain high face fidelity. Addressing these limitations, we introduce InstantID, a powerful diffusion model-based solution. Our plug-and-play module adeptly handles image personalization in various styles using just a single facial image, while ensuring high fidelity. To achieve this, we design a novel IdentityNet by imposing strong semantic and weak spatial conditions, integrating facial and landmark images with textual prompts to steer the image generation. InstantID demonstrates exceptional performance and efficiency, proving highly beneficial in real-world applications where identity preservation is paramount. Moreover, our work seamlessly integrates with popular pre-trained text-to-image diffusion models like SD1.5 and SDXL, serving as an adaptable plugin. Our codes and pre-trained checkpoints will be available at https://github.com/InstantID/InstantID.
Experimental Assessment of Containers Running on Top of Virtual Machines
Venue: arXiv cs.DC 2024
Year: 2024

Over the past two decades, the cloud computing paradigm has gradually attracted more popularity due to its efficient resource usage and simple service access model. Virtualization technology is the fundamental element of cloud computing that brings several benefits to cloud users and providers, such as workload isolation, energy efficiency, server consolidation, and cost reduction. This paper examines the combination of operating system-level virtualization (containers) and hardware-level virtualization (virtual machines). To this end, the performance of containers running on top of virtual machines is experimentally compared with standalone virtual machines and containers based on different hardware resources, including the processor, main memory, disk, and network in a real testbed by running the most commonly used benchmarks. Paravirtualization and full virtualization as well as type 1 and type 2 hypervisors are covered in this study. In addition, three prevalent containerization platforms are examined.
Must: Maximizing Latent Capacity of Spatial Transcriptomics Data
Venue: arXiv cs.CE 2024
Year: 2024

Spatial transcriptomics (ST) technologies have revolutionized the study of gene expression patterns in tissues by providing multimodality data in transcriptomic, spatial, and morphological, offering opportunities for understanding tissue biology beyond transcriptomics. However, we identify the modality bias phenomenon in ST data species, i.e., the inconsistent contribution of different modalities to the labels leads to a tendency for the analysis methods to retain the information of the dominant modality. How to mitigate the adverse effects of modality bias to satisfy various downstream tasks remains a fundamental challenge. This paper introduces Multiple-modality Structure Transformation, named MuST, a novel methodology to tackle the challenge. MuST integrates the multi-modality information contained in the ST data effectively into a uniform latent space to provide a foundation for all the downstream tasks. It learns intrinsic local structures by topology discovery strategy and topology fusion loss function to solve the inconsistencies among different modalities. Thus, these topology-based and deep learning techniques provide a solid foundation for a variety of analytical tasks while coordinating different modalities. The effectiveness of MuST is assessed by performance metrics and biological significance. The results show that it outperforms existing state-of-the-art methods with clear advantages in the precision of identifying and preserving structures of tissues and biomarkers. MuST offers a versatile toolkit for the intricate analysis of complex biological systems.
Exploiting GPT-4 Vision for Zero-shot Point Cloud Understanding
Venue: arXiv cs.CV 2024
Year: 2024

In this study, we tackle the challenge of classifying the object category in point clouds, which previous works like PointCLIP struggle to address due to the inherent limitations of the CLIP architecture. Our approach leverages GPT-4 Vision (GPT-4V) to overcome these challenges by employing its advanced generative abilities, enabling a more adaptive and robust classification process. We adapt the application of GPT-4V to process complex 3D data, enabling it to achieve zero-shot recognition capabilities without altering the underlying model architecture. Our methodology also includes a systematic strategy for point cloud image visualization, mitigating domain gap and enhancing GPT-4V's efficiency. Experimental validation demonstrates our approach's superiority in diverse scenarios, setting a new benchmark in zero-shot point cloud classification.
Cascaded Cross-Modal Transformer for Audio-Textual Classification
Venue: arXiv cs.CL 2024
Year: 2024

Speech classification tasks often require powerful language understanding models to grasp useful features, which becomes problematic when limited training data is available. To attain superior classification performance, we propose to harness the inherent value of multimodal representations by transcribing speech using automatic speech recognition (ASR) models and translating the transcripts into different languages via pretrained translation models. We thus obtain an audio-textual (multimodal) representation for each data sample. Subsequently, we combine language-specific Bidirectional Encoder Representations from Transformers (BERT) with Wav2Vec2.0 audio features via a novel cascaded cross-modal transformer (CCMT). Our model is based on two cascaded transformer blocks. The first one combines text-specific features from distinct languages, while the second one combines acoustic features with multilingual features previously learned by the first transformer block. We employed our system in the Requests Sub-Challenge of the ACM Multimedia 2023 Computational Paralinguistics Challenge. CCMT was declared the winning solution, obtaining an unweighted average recall (UAR) of 65.41% and 85.87% for complaint and request detection, respectively. Moreover, we applied our framework on the Speech Commands v2 and HarperValleyBank dialog data sets, surpassing previous studies reporting results on these benchmarks. Our code is freely available for download at: https://github.com/ristea/ccmt.
PyTester: Deep Reinforcement Learning for Text-to-Testcase Generation
Venue: arXiv cs.SE 2024
Year: 2024

Test-driven development (TDD) is a widely-employed software development practice that mandates writing test cases based on requirements before writing the actual code. While writing test cases is the centerpiece of TDD, it is time-consuming, expensive, and often shunned by developers. To address these issues associated with TDD, automated test case generation approaches have recently been investigated. Such approaches take source code as input, but not the requirements. Therefore, existing work does not fully support true TDD, as actual code is required to generate test cases. In addition, current deep learning-based test case generation approaches are trained with one learning objective, i.e., to generate test cases that are exactly matched with the ground-truth test cases. However, such approaches may limit the model's ability to generate different yet correct test cases. In this paper, we introduce PyTester, a Text-to-Testcase generation approach that can automatically generate syntactically correct, executable, complete, and effective test cases while being aligned with a given natural language requirement. We evaluate PyTester on the public APPS benchmark dataset, and the results show that our Deep RL approach enables PyTester, a small language model, to outperform much larger language models like GPT3.5, StarCoder, and InCoder. Our findings suggest that future research could consider improving small over large LMs for better resource efficiency by integrating the SE domain knowledge into the design of reinforcement learning architecture.
Multimodal Crowd Counting with Pix2Pix GANs
Venue: arXiv cs.CV 2024
Year: 2024

Most state-of-the-art crowd counting methods use color (RGB) images to learn the density map of the crowd. However, these methods often struggle to achieve higher accuracy in densely crowded scenes with poor illumination. Recently, some studies have reported improvement in the accuracy of crowd counting models using a combination of RGB and thermal images. Although multimodal data can lead to better predictions, multimodal data might not be always available beforehand. In this paper, we propose the use of generative adversarial networks (GANs) to automatically generate thermal infrared (TIR) images from color (RGB) images and use both to train crowd counting models to achieve higher accuracy. We use a Pix2Pix GAN network first to translate RGB images to TIR images. Our experiments on several state-of-the-art crowd counting models and benchmark crowd datasets report significant improvement in accuracy.
E3x: $\mathrm{E}(3)$-Equivariant Deep Learning Made Easy
Venue: arXiv cs.LG 2024
Year: 2024

This work introduces E3x, a software package for building neural networks that are equivariant with respect to the Euclidean group $\mathrm{E}(3)$, consisting of translations, rotations, and reflections of three-dimensional space. Compared to ordinary neural networks, $\mathrm{E}(3)$-equivariant models promise benefits whenever input and/or output data are quantities associated with three-dimensional objects. This is because the numeric values of such quantities (e.g. positions) typically depend on the chosen coordinate system. Under transformations of the reference frame, the values change predictably, but the underlying rules can be difficult to learn for ordinary machine learning models. With built-in $\mathrm{E}(3)$-equivariance, neural networks are guaranteed to satisfy the relevant transformation rules exactly, resulting in superior data efficiency and accuracy. The code for E3x is available from https://github.com/google-research/e3x, detailed documentation and usage examples can be found on https://e3x.readthedocs.io.
Model checking strategy-controlled systems in rewriting logic
Venue: arXiv cs.LO 2024
Year: 2024

Rewriting logic and its implementation Maude are an expressive framework for the formal specification and verification of software and other kinds of systems. Concurrency is naturally represented by nondeterministic local transformations produced by the application of rewriting rules over algebraic terms in an equational theory. Some aspects of the global behavior of the systems or additional constraints sometimes require restricting this nondeterminism. Rewriting strategies are used as a higher-level and modular resource to cleanly capture these requirements, which can be easily expressed in Maude with an integrated strategy language. However, strategy-aware specifications cannot be verified with the builtin LTL model checker, making strategies less useful and attractive. In this paper, we discuss model checking for strategy-controlled systems, and present a strategy-aware extension of the Maude LTL model checker. The expressivity of the strategy language is discussed in relation to model checking, the model checker is illustrated with multiple application examples, and its performance is compared.
MLAD: A Unified Model for Multi-system Log Anomaly Detection
Venue: arXiv cs.SE 2024
Year: 2024

In spite of the rapid advancements in unsupervised log anomaly detection techniques, the current mainstream models still necessitate specific training for individual system datasets, resulting in costly procedures and limited scalability due to dataset size, thereby leading to performance bottlenecks. Furthermore, numerous models lack cognitive reasoning capabilities, posing challenges in direct transferability to similar systems for effective anomaly detection. Additionally, akin to reconstruction networks, these models often encounter the "identical shortcut" predicament, wherein the majority of system logs are classified as normal, erroneously predicting normal classes when confronted with rare anomaly logs due to reconstruction errors. To address the aforementioned issues, we propose MLAD, a novel anomaly detection model that incorporates semantic relational reasoning across multiple systems. Specifically, we employ Sentence-bert to capture the similarities between log sequences and convert them into highly-dimensional learnable semantic vectors. Subsequently, we revamp the formulas of the Attention layer to discern the significance of each keyword in the sequence and model the overall distribution of the multi-system dataset through appropriate vector space diffusion. Lastly, we employ a Gaussian mixture model to highlight the uncertainty of rare words pertaining to the "identical shortcut" problem, optimizing the vector space of the samples using the maximum expectation model. Experiments on three real-world datasets demonstrate the superiority of MLAD.
Learning Explainable and Better Performing Representations of POMDP Strategies
Venue: arXiv cs.AI 2024
Year: 2024

Strategies for partially observable Markov decision processes (POMDP) typically require memory. One way to represent this memory is via automata. We present a method to learn an automaton representation of a strategy using a modification of the L*-algorithm. Compared to the tabular representation of a strategy, the resulting automaton is dramatically smaller and thus also more explainable. Moreover, in the learning process, our heuristics may even improve the strategy's performance. In contrast to approaches that synthesize an automaton directly from the POMDP thereby solving it, our approach is incomparably more scalable.
Empirical Evidence for the Fragment level Understanding on Drug Molecular Structure of LLMs
Venue: arXiv cs.LG 2024
Year: 2024

AI for drug discovery has been a research hotspot in recent years, and SMILES-based language models has been increasingly applied in drug molecular design. However, no work has explored whether and how language models understand the chemical spatial structure from 1D sequences. In this work, we pre-train a transformer model on chemical language and fine-tune it toward drug design objectives, and investigate the correspondence between high-frequency SMILES substrings and molecular fragments. The results indicate that language models can understand chemical structures from the perspective of molecular fragments, and the structural knowledge learned through fine-tuning is reflected in the high-frequency SMILES substrings generated by the model.
Strategies, model checking and branching-time properties in Maude
Venue: arXiv cs.LO 2024
Year: 2024

Rewriting logic and its implementation Maude are a natural and expressive framework for the specification of concurrent systems and logics. Its nondeterministic local transformations are described by rewriting rules, which can be controlled at a higher level using a builtin strategy language added to Maude~3. This specification resource would not be of much interest without tools to analyze their models, so in a previous work, we extended the Maude LTL model checker to verify strategy-controlled systems. In this paper, CTL* and $μ$-calculus are added to the repertoire of supported logics, after discussing which adaptations are needed for branching-time properties. The new extension relies on some external model checkers that are exposed the Maude models through general and efficient connections, profitable for future extensions and further applications. The performance of these model checkers is compared.
Data vs. Model Machine Learning Fairness Testing: An Empirical Study
Venue: arXiv cs.LG 2024
Year: 2024

Although several fairness definitions and bias mitigation techniques exist in the literature, all existing solutions evaluate fairness of Machine Learning (ML) systems after the training stage. In this paper, we take the first steps towards evaluating a more holistic approach by testing for fairness both before and after model training. We evaluate the effectiveness of the proposed approach and position it within the ML development lifecycle, using an empirical analysis of the relationship between model dependent and independent fairness metrics. The study uses 2 fairness metrics, 4 ML algorithms, 5 real-world datasets and 1600 fairness evaluation cycles. We find a linear relationship between data and model fairness metrics when the distribution and the size of the training data changes. Our results indicate that testing for fairness prior to training can be a ``cheap'' and effective means of catching a biased data collection process early; detecting data drifts in production systems and minimising execution of full training cycles thus reducing development time and costs.
A Pragmatical Approach to Anomaly Detection Evaluation in Edge Cloud Systems
Venue: arXiv cs.NI 2024
Year: 2024

Anomaly detection (AD) has been recently employed in the context of edge cloud computing, e.g., for intrusion detection and identification of performance issues. However, state-of-the-art anomaly detection procedures do not systematically consider restrictions and performance requirements inherent to the edge, such as system responsiveness and resource consumption. In this paper, we attempt to investigate the performance of change-point based detectors, i.e., a class of lightweight and accurate AD methods, in relation to the requirements of edge cloud systems. Firstly, we review the theoretical properties of two major categories of change point approaches, i.e., Bayesian and cumulative sum (CUSUM), also discussing their suitability for edge systems. Secondly, we introduce a novel experimental methodology and apply it over two distinct edge cloud test-beds to evaluate the performance of such mechanisms in real-world edge environments. Our experimental results reveal important insights and trade-offs for the applicability and the online performance of the selected change point detectors.
How Social Media Big Data Can Improve Suicide Prevention
Venue: arXiv cs.CY 2024
Year: 2024

In the light of increasing clues on social media impact on self-harm and suicide risks, there is still no evidence on who are and how factually engaged in suicide-related online behaviors. This study reports new findings of high-performance supercomputing investigation of publicly accessible big data sourced from one of the world-largest social networking site. Three-month supercomputer searching resulted in 570,156 young adult users who consumed suicide-related information on social media. Most of them were 21-24 year olds with higher share of females (58%) of predominantly younger age. Every eight user was alarmingly engrossed with up to 15 suicide-related online groups. Evidently, suicide groups on social media are highly underrated public health issue that might weaken the prevention efforts. Suicide prevention strategies that target social media users must be implemented extensively. While major gap in functional understanding of technologies relevance for use in public mental health still exists, current findings act for better understanding digital technologies utility for translational advance and offer relevant evidence-based framework for improving suicide prevention in general population.
Low-light Stereo Image Enhancement and De-noising in the Low-frequency Information Enhanced Image Space
Venue: arXiv cs.CV 2024
Year: 2024

Unlike single image task, stereo image enhancement can use another view information, and its key stage is how to perform cross-view feature interaction to extract useful information from another view. However, complex noise in low-light image and its impact on subsequent feature encoding and interaction are ignored by the existing methods. In this paper, a method is proposed to perform enhancement and de-noising simultaneously. First, to reduce unwanted noise interference, a low-frequency information enhanced module (IEM) is proposed to suppress noise and produce a new image space. Additionally, a cross-channel and spatial context information mining module (CSM) is proposed to encode long-range spatial dependencies and to enhance inter-channel feature interaction. Relying on CSM, an encoder-decoder structure is constructed, incorporating cross-view and cross-scale feature interactions to perform enhancement in the new image space. Finally, the network is trained with the constraints of both spatial and frequency domain losses. Extensive experiments on both synthesized and real datasets show that our method obtains better detail recovery and noise removal compared with state-of-the-art methods. In addition, a real stereo image enhancement dataset is captured with stereo camera ZED2. The code and dataset are publicly available at: https://www.github.com/noportraits/LFENet.