OpenRaiser/Intern-Atlas huggingface dataset:
Physics-Informed Machine Learning On Polar Ice: A Survey
Venue: arXiv cs.LG 2024
Year: 2024

The mass loss of the polar ice sheets contributes considerably to ongoing sea-level rise and changing ocean circulation, leading to coastal flooding and risking the homes and livelihoods of tens of millions of people globally. To address the complex problem of ice behavior, physical models and data-driven models have been proposed in the literature. Although traditional physical models can guarantee physically meaningful results, they have limitations in producing high-resolution results. On the other hand, data-driven approaches require large amounts of high-quality and labeled data, which is rarely available in the polar regions. Hence, as a promising framework that leverages the advantages of physical models and data-driven methods, physics-informed machine learning (PIML) has been widely studied in recent years. In this paper, we review the existing algorithms of PIML, provide our own taxonomy based on the methods of combining physics and data-driven approaches, and analyze the advantages of PIML in the aspects of accuracy and efficiency. Further, our survey discusses some current challenges and highlights future opportunities, including PIML on sea ice studies, PIML with different combination methods and backbone networks, and neural operator methods.
One-Stage Open-Vocabulary Temporal Action Detection Leveraging Temporal Multi-scale and Action Label Features
Venue: arXiv cs.CV 2024
Year: 2024

Open-vocabulary Temporal Action Detection (Open-vocab TAD) is an advanced video analysis approach that expands Closed-vocabulary Temporal Action Detection (Closed-vocab TAD) capabilities. Closed-vocab TAD is typically confined to localizing and classifying actions based on a predefined set of categories. In contrast, Open-vocab TAD goes further and is not limited to these predefined categories. This is particularly useful in real-world scenarios where the variety of actions in videos can be vast and not always predictable. The prevalent methods in Open-vocab TAD typically employ a 2-stage approach, which involves generating action proposals and then identifying those actions. However, errors made during the first stage can adversely affect the subsequent action identification accuracy. Additionally, existing studies face challenges in handling actions of different durations owing to the use of fixed temporal processing methods. Therefore, we propose a 1-stage approach consisting of two primary modules: Multi-scale Video Analysis (MVA) and Video-Text Alignment (VTA). The MVA module captures actions at varying temporal resolutions, overcoming the challenge of detecting actions with diverse durations. The VTA module leverages the synergy between visual and textual modalities to precisely align video segments with corresponding action labels, a critical step for accurate action identification in Open-vocab scenarios. Evaluations on widely recognized datasets THUMOS14 and ActivityNet-1.3, showed that the proposed method achieved superior results compared to the other methods in both Open-vocab and Closed-vocab settings. This serves as a strong demonstration of the effectiveness of the proposed method in the TAD task.
An Extensive Survey of Digital Image Steganography: State of the Art
Venue: arXiv cs.ET 2024
Year: 2024

The need to protect sensitive information privacy duringinformation exchange over the internet/intranet has led towider adoption of cryptography and steganography. The cryptography approaches convert the information into an unreadable format however draws the attention of cryptanalyst owing to the uncommon random nature flow of the bytes when viewing the flowing structured bytes on a computer. While steganography, in contrast, conceals the very existence of covert communication using digital media. Although any digital media (text, image, video, audio) can covey the sensitive information, the media with higher redundant bits are more favorable for embedding the sensitive information without distorting the media. Digital images are majorly used in conveying sensitive information compared to others owing to their higher rate of tolerating distortions, highly available, smaller sizes with high redundant bits. However, the need for maximizing the redundancy bits for the optimum embedding of secret information has been a paramount issue due to the imperceptibility prerequisite which deteriorates with an increase in payload thus, resulting in a tradeoff. This has limited steganography to only applications with lower payload requirements, thus limiting the adoption for wider deployment. This paper critically analyzes the current steganographic techniques, recent trends, and challenges.
Causal Perception Inspired Representation Learning for Trustworthy Image Quality Assessment
Venue: arXiv cs.CV 2024
Year: 2024

Despite great success in modeling visual perception, deep neural network based image quality assessment (IQA) still remains unreliable in real-world applications due to its vulnerability to adversarial perturbations and the inexplicit black-box structure. In this paper, we propose to build a trustworthy IQA model via Causal Perception inspired Representation Learning (CPRL), and a score reflection attack method for IQA model. More specifically, we assume that each image is composed of Causal Perception Representation (CPR) and non-causal perception representation (N-CPR). CPR serves as the causation of the subjective quality label, which is invariant to the imperceptible adversarial perturbations. Inversely, N-CPR presents spurious associations with the subjective quality label, which may significantly change with the adversarial perturbations. To extract the CPR from each input image, we develop a soft ranking based channel-wise activation function to mediate the causally sufficient (beneficial for high prediction accuracy) and necessary (beneficial for high robustness) deep features, and based on intervention employ minimax game to optimize. Experiments on four benchmark databases show that the proposed CPRL method outperforms many state-of-the-art adversarial defense methods and provides explicit model interpretation.
Seeing Through the Clouds: Cloud Gap Imputation with Prithvi Foundation Model
Venue: arXiv cs.CV 2024
Year: 2024

Filling cloudy pixels in multispectral satellite imagery is essential for accurate data analysis and downstream applications, especially for tasks which require time series data. To address this issue, we compare the performance of a foundational Vision Transformer (ViT) model with a baseline Conditional Generative Adversarial Network (CGAN) model for missing value imputation in time series of multispectral satellite imagery. We randomly mask time series of satellite images using real-world cloud masks and train each model to reconstruct the missing pixels. The ViT model is fine-tuned from a pretrained model, while the CGAN is trained from scratch. Using quantitative evaluation metrics such as structural similarity index and mean absolute error as well as qualitative visual analysis, we assess imputation accuracy and contextual preservation.
Quantum Cloud Computing: Trends and Challenges
Venue: arXiv cs.DC 2024
Year: 2024

Quantum computing (QC) is a new paradigm that will revolutionize various areas of computing, especially cloud computing. QC, still in its infancy, is a costly technology capable of operating in highly isolated environments due to its rapid response to environmental factors. For this reason, it is still a challenging technology for researchers to reach. Integrating QC into an isolated remote server, like a cloud, and making it available to users can overcome these problems. Furthermore, experts predict that QC, with its ability to swiftly resolve complex and computationally intensive operations, will offer significant benefits in systems that process large amounts of data, like cloud computing. This article presents the vision and challenges for the quantum cloud computing (QCC) paradigm that will emerge with the integration of quantum and cloud computing. Next, we present the advantages of QC over classical computing applications. We analyze the effects of QC on cloud systems, such as cost, security, and scalability. Besides all of these advantages, we highlight research gaps in QCC, such as qubit stability and efficient resource allocation. This article identifies QCC's advantages and challenges for future research, highlighting research gaps.
COTS: Connected OpenAPI Test Synthesis for RESTful Applications
Venue: arXiv cs.SE 2024
Year: 2024

We present a novel model-driven approach for testing RESTful applications. We introduce a (i) domain-specific language for OpenAPI specifications and (ii) a tool to support our methodology. Our DSL is inspired by session types and enables the modelling of communication protocols between a REST client and server. Our tool, dubbed COTS, generates (randomised) model-based test executions and reports software defects. We evaluate the effectiveness of our approach by applying it to test several open source applications. Our findings indicate that our methodology can identify nuanced defects in REST APIs and achieve comparable or superior code coverage when compared to much larger handcrafted test suites.
Physical Non-inertial Poser (PNP): Modeling Non-inertial Effects in Sparse-inertial Human Motion Capture
Venue: arXiv cs.GR 2024
Year: 2024

Existing inertial motion capture techniques use the human root coordinate frame to estimate local poses and treat it as an inertial frame by default. We argue that when the root has linear acceleration or rotation, the root frame should be considered non-inertial theoretically. In this paper, we model the fictitious forces that are non-neglectable in a non-inertial frame by an auto-regressive estimator delicately designed following physics. With the fictitious forces, the force-related IMU measurement (accelerations) can be correctly compensated in the non-inertial frame and thus Newton's laws of motion are satisfied. In this case, the relationship between the accelerations and body motions is deterministic and learnable, and we train a neural network to model it for better motion capture. Furthermore, to train the neural network with synthetic data, we develop an IMU synthesis by simulation strategy to better model the noise model of IMU hardware and allow parameter tuning to fit different hardware. This strategy not only establishes the network training with synthetic data but also enables calibration error modeling to handle bad motion capture calibration, increasing the robustness of the system. Code is available at https://xinyu-yi.github.io/PNP/.
SEArch: an execution infrastructure for service-based software systems
Venue: arXiv cs.SE 2024
Year: 2024

The shift from monolithic applications to composition of distributed software initiated in the early twentieth, is based on the vision of software-as-service. This vision, found in many technologies such as RESTful APIs, advocates globally available services cooperating through an infrastructure providing (access to) distributed computational resources. Choreographies can support this vision by abstracting away local computation and rendering interoperability with message-passing: cooperation is achieved by sending and receiving messages. Following this choreographic paradigm, we develop SEArch, after Service Execution Architecture, a language-independent execution infrastructure capable of performing transparent dynamic reconfiguration of software artefacts. Choreographic mechanisms are used in SEArch to specify interoperability contracts, thus providing the support needed for automatic discovery and binding of services at runtime.
Attacking Bayes: On the Adversarial Robustness of Bayesian Neural Networks
Venue: arXiv cs.LG 2024
Year: 2024

Adversarial examples have been shown to cause neural networks to fail on a wide range of vision and language tasks, but recent work has claimed that Bayesian neural networks (BNNs) are inherently robust to adversarial perturbations. In this work, we examine this claim. To study the adversarial robustness of BNNs, we investigate whether it is possible to successfully break state-of-the-art BNN inference methods and prediction pipelines using even relatively unsophisticated attacks for three tasks: (1) label prediction under the posterior predictive mean, (2) adversarial example detection with Bayesian predictive uncertainty, and (3) semantic shift detection. We find that BNNs trained with state-of-the-art approximate inference methods, and even BNNs trained with Hamiltonian Monte Carlo, are highly susceptible to adversarial attacks. We also identify various conceptual and experimental errors in previous works that claimed inherent adversarial robustness of BNNs and conclusively demonstrate that BNNs and uncertainty-aware Bayesian prediction pipelines are not inherently robust against adversarial attacks.
VimTS: A Unified Video and Image Text Spotter for Enhancing the Cross-domain Generalization
Venue: arXiv cs.CV 2024
Year: 2024

Text spotting, a task involving the extraction of textual information from image or video sequences, faces challenges in cross-domain adaption, such as image-to-image and image-to-video generalization. In this paper, we introduce a new method, termed VimTS, which enhances the generalization ability of the model by achieving better synergy among different tasks. Typically, we propose a Prompt Queries Generation Module and a Tasks-aware Adapter to effectively convert the original single-task model into a multi-task model suitable for both image and video scenarios with minimal additional parameters. The Prompt Queries Generation Module facilitates explicit interaction between different tasks, while the Tasks-aware Adapter helps the model dynamically learn suitable features for each task. Additionally, to further enable the model to learn temporal information at a lower cost, we propose a synthetic video text dataset (VTD-368k) by leveraging the Content Deformation Fields (CoDeF) algorithm. Notably, our method outperforms the state-of-the-art method by an average of 2.6% in six cross-domain benchmarks such as TT-to-IC15, CTW1500-to-TT, and TT-to-CTW1500. For video-level cross-domain adaption, our method even surpasses the previous end-to-end video spotting method in ICDAR2015 video and DSText v2 by an average of 5.5% on the MOTA metric, using only image-level data. We further demonstrate that existing Large Multimodal Models exhibit limitations in generating cross-domain scene text spotting, in contrast to our VimTS model which requires significantly fewer parameters and data. The code and datasets will be made available at the https://VimTextSpotter.github.io.
A Comprehensive Analysis of Pegasus Spyware and Its Implications for Digital Privacy and Security
Venue: arXiv cs.CR 2024
Year: 2024

This paper comprehensively analyzes the Pegasus spyware and its implications for digital privacy and security. The Israeli cyber intelligence company NSO Group's Pegasus has gained recognition as a potent surveillance tool capable of hacking into smartphones and extracting data without the user's knowledge [49], [50]. The research emphasizes the technical aspects of this spyware, its deployment methods, and the controversies surrounding its use. The research also emphasizes the growing worries surrounding digital privacy and security as a result of the prevalent use of advanced spyware. By delving into legal, ethical, and policy issues, the objective of this study is to deliver a holistic understanding of the challenges posed by Pegasus and similar spyware tools. Through a comprehensive examination of the subject, the paper presents potential solutions to mitigate the threats and protect users from invasive surveillance techniques.
Collaborative Control Method of Transit Signal Priority Based on Cooperative Game and Reinforcement Learning
Venue: arXiv cs.GT 2024
Year: 2024

To address the low efficiency in priority signal control within intelligent transportation systems, this study introduces a novel eight-phase priority signal control method, CBQL-TSP, leveraging a hybrid decision-making framework that integrates cooperative game theory and reinforcement learning. This approach conceptualizes the allocation of bus signal priorities as a multi-objective decision-making problem across an eight-phase signal sequence, differentiating between priority and non-priority phases. It employs a cooperative game model to facilitate this differentiation. The developed hybrid decision-making algorithm, CBQL, effectively tackles the multi-objective decision-making challenges inherent in the eight-phase signal sequence. By computing the Shapley value function, it quantifies the marginal contributions of each participant, which in turn inform the construction of a state transition probability equation based on Shapley value ratios. Compared to conventional control methods, the CBQL-TSP method not only upholds the fairness principles of cooperative game theory but also harnesses the adaptive learning capabilities of Q-Learning. This enables dynamic adjustments to signal timing in response to real-time traffic conditions, significantly enhancing the flexibility and efficiency of priority signal control.
Harmonic LLMs are Trustworthy
Venue: arXiv cs.LG 2024
Year: 2024

We introduce an intuitive method to test the robustness (stability and explainability) of any black-box LLM in real-time via its local deviation from harmoniticity, denoted as $γ$. To the best of our knowledge this is the first completely model-agnostic and unsupervised method of measuring the robustness of any given response from an LLM, based upon the model itself conforming to a purely mathematical standard. To show general application and immediacy of results, we measure $γ$ in 10 popular LLMs (ChatGPT, Claude-2.1, Claude3.0, GPT-4, GPT-4o, Smaug-72B, Mixtral-8x7B, Llama2-7B, Mistral-7B and MPT-7B) across thousands of queries in three objective domains: WebQA, ProgrammingQA, and TruthfulQA. Across all models and domains tested, human annotation confirms that $γ\to 0$ indicates trustworthiness, and conversely searching higher values of $γ$ easily exposes examples of hallucination, a fact that enables efficient adversarial prompt generation through stochastic gradient ascent in $γ$. The low-$γ$ leaders among the models in the respective domains are GPT-4o, GPT-4, and Smaug-72B, providing evidence that mid-size open-source models can win out against large commercial models.
The lazy (NTK) and rich ($μ$P) regimes: a gentle tutorial
Venue: arXiv cs.LG 2024
Year: 2024

A central theme of the modern machine learning paradigm is that larger neural networks achieve better performance on a variety of metrics. Theoretical analyses of these overparameterized models have recently centered around studying very wide neural networks. In this tutorial, we provide a nonrigorous but illustrative derivation of the following fact: in order to train wide networks effectively, there is only one degree of freedom in choosing hyperparameters such as the learning rate and the size of the initial weights. This degree of freedom controls the richness of training behavior: at minimum, the wide network trains lazily like a kernel machine, and at maximum, it exhibits feature learning in the active $μ$P regime. In this paper, we explain this richness scale, synthesize recent research results into a coherent whole, offer new perspectives and intuitions, and provide empirical evidence supporting our claims. In doing so, we hope to encourage further study of the richness scale, as it may be key to developing a scientific theory of feature learning in practical deep neural networks.
Cantera-Based Python Computer Program for Solving Steam Power Cycles with Superheating
Venue: arXiv cs.CE 2024
Year: 2024

One of the main sources of electricity generation is power plants that use water (steam) to rotate turbines, which drive large electric generators. The steam can be generated from renewable or non-renewable energy sources, such as geothermal energy and nuclear fuels. Having an analysis tool for modeling the performance of such steam power plants can greatly help in reaching optimum designs, leading to less fuel consumption, reduced pollution, and cheaper electricity. It is further advantageous if such modeling tool is free to access, does not require many inputs from the user, and gives results in a very short time. These remarks establish a motivation for the current study. This article documents a computer code written in the Python programming language for numerically analysing the main processes in a steam power cycle with superheating. The code utilizes built-in thermodynamic properties for water in the open-source software package "Cantera". A validation case with a benchmarking example in the literature using an independent source of water properties suggests that the developed code is correct. The code can be viewed as an extension to the Python examples for thermodynamic and power generation applications. Cantera can handle both subcritical and supercritical types of superheating. In the subcritical superheating, the steam absolute pressure does not exceed 220.9 bar. In the supercritical superheating, water becomes in a special condition called supercritical fluid, with absolute pressures above 220.9 bar.
Research and application of artificial intelligence based webshell detection model: A literature review
Venue: arXiv cs.CR 2024
Year: 2024

Webshell, as the "culprit" behind numerous network attacks, is one of the research hotspots in the field of cybersecurity. However, the complexity, stealthiness, and confusing nature of webshells pose significant challenges to the corresponding detection schemes. With the rise of Artificial Intelligence (AI) technology, researchers have started to apply different intelligent algorithms and neural network architectures to the task of webshell detection. However, the related research still lacks a systematic and standardized methodological process, which is confusing and redundant. Therefore, following the development timeline, we carefully summarize the progress of relevant research in this field, dividing it into three stages: Start Stage, Initial Development Stage, and In-depth Development Stage. We further elaborate on the main characteristics and core algorithms of each stage. In addition, we analyze the pain points and challenges that still exist in this field and predict the future development trend of this field from our point of view. To the best of our knowledge, this is the first review that details the research related to AI-based webshell detection. It is also hoped that this paper can provide detailed technical information for more researchers interested in AI-based webshell detection tasks.
Graph Neural Network Approach to Semantic Type Detection in Tables
Venue: arXiv cs.LG 2024
Year: 2024

This study addresses the challenge of detecting semantic column types in relational tables, a key task in many real-world applications. While language models like BERT have improved prediction accuracy, their token input constraints limit the simultaneous processing of intra-table and inter-table information. We propose a novel approach using Graph Neural Networks (GNNs) to model intra-table dependencies, allowing language models to focus on inter-table information. Our proposed method not only outperforms existing state-of-the-art algorithms but also offers novel insights into the utility and functionality of various GNN types for semantic type detection. The code is available at https://github.com/hoseinzadeehsan/GAIT