OpenRaiser/Intern-Atlas huggingface dataset:
A Transformer-Based Framework for Payload Malware Detection and Classification
Venue: arXiv cs.CR 2024
Year: 2024

As malicious cyber threats become more sophisticated in breaching computer networks, the need for effective intrusion detection systems (IDSs) becomes crucial. Techniques such as Deep Packet Inspection (DPI) have been introduced to allow IDSs analyze the content of network packets, providing more context for identifying potential threats. IDSs traditionally rely on using anomaly-based and signature-based detection techniques to detect unrecognized and suspicious activity. Deep learning techniques have shown great potential in DPI for IDSs due to their efficiency in learning intricate patterns from the packet content being transmitted through the network. In this paper, we propose a revolutionary DPI algorithm based on transformers adapted for the purpose of detecting malicious traffic with a classifier head. Transformers learn the complex content of sequence data and generalize them well to similar scenarios thanks to their self-attention mechanism. Our proposed method uses the raw payload bytes that represent the packet contents and is deployed as man-in-the-middle. The payload bytes are used to detect malicious packets and classify their types. Experimental results on the UNSW-NB15 and CIC-IOT23 datasets demonstrate that our transformer-based model is effective in distinguishing malicious from benign traffic in the test dataset, attaining an average accuracy of 79\% using binary classification and 72\% on the multi-classification experiment, both using solely payload bytes.
A Comprehensive Overview of the Lebesgue Differentiation Theorem in Coq
Venue: arXiv cs.LO 2024
Year: 2024

Formalization of real analysis offers a chance to rebuild traditional proofs of important theorems as unambiguous theories that can be interactively explored. This paper provides a comprehensive overview of the Lebesgue Differentiation Theorem formalized in the Coq proof assistant, from which the first Fundamental Theorem of Calculus (FTC) for the Lebesgue integral is obtained as a corollary. Proving the first FTC in this way has the advantage of decomposing into loosely-coupled theories of moderate size and of independent interest that lend themselves well to incremental and collaborative development. We explain how we formalize all the topological constructs and all the standard lemmas needed to eventually relate the definitions of derivability and of Lebesgue integration of MathComp-Analysis, a formalization of analysis developed on top of the Mathematical Components library. In the course of this experiment, we substantially enrich MathComp-Analysis and even devise a new proof for Urysohn's lemma.
The Dimensions of the Hulls of Conorm Codes from Algebraic Geometry Codes
Venue: arXiv cs.IT 2024
Year: 2024

Chara et al. introduced conorm codes defined over algebraic geometry codes, but the hulls of conorm codes were not determined yet. In this paper, we study the dimension of the hull of conorm codes using the method introduced by Camps et al. For an algebraic geometry code $\mathcal{C}:=C_\mathscr{L}(D, G)$, we consider the divisor $\gcd(G, H)$, where $H$ is the divisor satisfying \[C_\mathscr{L}(D, G)^\perp=C_\mathscr{L}(D, H).\] Given an extension $F'/\mathbb{F}_{q^t}$ of an algebraic function field $F/\mathbb{F}_q$, we assume that the divisor $\gcd(G, H)$ is non-special. If the degree of $\gcd(G, H)$ is greater than $2g-2+{t\over [F':F]}°\text{Diff}(F'/F)$, then we have determined the exact dimension of the hull of the conorm of $\mathcal{C}$. If not, we have determined the lower bound of the dimension of the hull of the conorm of $\mathcal{C}$. We provide some examples for the dimension of the hull of certain conorm codes of AG codes defined over a rational function field.
An Experimentally Validated Feasible Quantum Protocol for Identity-Based Signature with Application to Secure Email Communication
Venue: arXiv cs.CR 2024
Year: 2024

Digital signatures are one of the simplest cryptographic building blocks that provide appealing security characteristics such as authenticity, unforgeability, and undeniability. In 1984, Shamir developed the first Identity-based signature (IBS) to simplify public key infrastructure and circumvent the need for certificates. It makes the process uncomplicated by enabling users to verify digital signatures using only the identifiers of signers, such as email, phone number, etc. Nearly all existing IBS protocols rely on several theoretical assumption-based hard problems. Unfortunately, these hard problems are unsafe and pose a hazard in the quantum realm. Thus, designing IBS algorithms that can withstand quantum attacks and ensure long-term security is an important direction for future research. Quantum cryptography (QC) is one such approach. In this paper, we propose an IBS based on QC. Our scheme's security is based on the laws of quantum mechanics. It thereby achieves long-term security and provides resistance against quantum attacks. We verify the proposed design's correctness and feasibility by simulating it in a prototype quantum device and the IBM Qiskit quantum simulator. The implementation code in qiskit with Jupyternotebook is provided in the Annexure. Moreover, we discuss the application of our design in secure email communication.
Improving Out-of-Vocabulary Handling in Recommendation Systems
Venue: arXiv cs.IR 2024
Year: 2024

Recommendation systems (RS) are an increasingly relevant area for both academic and industry researchers, given their widespread impact on the daily online experiences of billions of users. One common issue in real RS is the cold-start problem, where users and items may not contain enough information to produce high-quality recommendations. This work focuses on a complementary problem: recommending new users and items unseen (out-of-vocabulary, or OOV) at training time. This setting is known as the inductive setting and is especially problematic for factorization-based models, which rely on encoding only those users/items seen at training time with fixed parameter vectors. Many existing solutions applied in practice are often naive, such as assigning OOV users/items to random buckets. In this work, we tackle this problem and propose approaches that better leverage available user/item features to improve OOV handling at the embedding table level. We discuss general-purpose plug-and-play approaches that are easily applicable to most RS models and improve inductive performance without negatively impacting transductive model performance. We extensively evaluate 9 OOV embedding methods on 5 models across 4 datasets (spanning different domains). One of these datasets is a proprietary production dataset from a prominent RS employed by a large social platform serving hundreds of millions of daily active users. In our experiments, we find that several proposed methods that exploit feature similarity using LSH consistently outperform alternatives on most model-dataset combinations, with the best method showing a mean improvement of 3.74% over the industry standard baseline in inductive performance. We release our code and hope our work helps practitioners make more informed decisions when handling OOV for their RS and further inspires academic research into improving OOV support in RS.
SGDM: Static-Guided Dynamic Module Make Stronger Visual Models
Venue: arXiv cs.CV 2024
Year: 2024

The spatial attention mechanism has been widely used to improve object detection performance. However, its operation is currently limited to static convolutions lacking content-adaptive features. This paper innovatively approaches from the perspective of dynamic convolution. We propose Razor Dynamic Convolution (RDConv) to address thetwo flaws in dynamic weight convolution, making it hard to implement in spatial mechanism: 1) it is computation-heavy; 2) when generating weights, spatial information is disregarded. Firstly, by using Razor Operation to generate certain features, we vastly reduce the parameters of the entire dynamic convolution operation. Secondly, we added a spatial branch inside RDConv to generate convolutional kernel parameters with richer spatial information. Embedding dynamic convolution will also bring the problem of sensitivity to high-frequency noise. We propose the Static-Guided Dynamic Module (SGDM) to address this limitation. By using SGDM, we utilize a set of asymmetric static convolution kernel parameters to guide the construction of dynamic convolution. We introduce the mechanism of shared weights in static convolution to solve the problem of dynamic convolution being sensitive to high-frequency noise. Extensive experiments illustrate that multiple different object detection backbones equipped with SGDM achieve a highly competitive boost in performance(e.g., +4% mAP with YOLOv5n on VOC and +1.7% mAP with YOLOv8n on COCO) with negligible parameter increase(i.e., +0.33M on YOLOv5n and +0.19M on YOLOv8n).
Sm-Nd Isotope Data Compilation from Geoscientific Literature Using an Automated Tabular Extraction Method
Venue: arXiv cs.DB 2024
Year: 2024

The rare earth elements Sm and Nd significantly address fundamental questions about crustal growth, such as its spatiotemporal evolution and the interplay between orogenesis and crustal accretion. Their relative immobility during high-grade metamorphism makes the Sm-Nd isotopic system crucial for inferring crustal formation times. Historically, data have been disseminated sporadically in the scientific literature due to complicated and costly sampling procedures, resulting in a fragmented knowledge base. However, the scattering of critical geoscience data across multiple publications poses significant challenges regarding human capital and time. In response, we present an automated tabular extraction method for harvesting tabular geoscience data. We collect 10,624 Sm-Nd data entries from 9,138 tables in over 20,000 geoscience publications using this method. We manually selected 2,118 data points from it to supplement our previously constructed global Sm-Nd dataset, increasing its sample count by over 20\%. Our automatic data collection methodology enhances the efficiency of data acquisition processes spanning various scientific domains. Furthermore, the constructed Sm-Nd isotopic dataset should motivate the research of classifying global orogenic belts.
Privacy-Preserving Distributed Nonnegative Matrix Factorization
Venue: arXiv cs.CR 2024
Year: 2024

Nonnegative matrix factorization (NMF) is an effective data representation tool with numerous applications in signal processing and machine learning. However, deploying NMF in a decentralized manner over ad-hoc networks introduces privacy concerns due to the conventional approach of sharing raw data among network agents. To address this, we propose a privacy-preserving algorithm for fully-distributed NMF that decomposes a distributed large data matrix into left and right matrix factors while safeguarding each agent's local data privacy. It facilitates collaborative estimation of the left matrix factor among agents and enables them to estimate their respective right factors without exposing raw data. To ensure data privacy, we secure information exchanges between neighboring agents utilizing the Paillier cryptosystem, a probabilistic asymmetric algorithm for public-key cryptography that allows computations on encrypted data without decryption. Simulation results conducted on synthetic and real-world datasets demonstrate the effectiveness of the proposed algorithm in achieving privacy-preserving distributed NMF over ad-hoc networks.
PIPNet3D: Interpretable Detection of Alzheimer in MRI Scans
Venue: arXiv cs.CV 2024
Year: 2024

Information from neuroimaging examinations is increasingly used to support diagnoses of dementia, e.g., Alzheimer's disease. While current clinical practice is mainly based on visual inspection and feature engineering, Deep Learning approaches can be used to automate the analysis and to discover new image-biomarkers. Part-prototype neural networks (PP-NN) are an alternative to standard blackbox models, and have shown promising results in general computer vision. PP-NN's base their reasoning on prototypical image regions that are learned fully unsupervised, and combined with a simple-to-understand decision layer. We present PIPNet3D, a PP-NN for volumetric images. We apply PIPNet3D to the clinical diagnosis of Alzheimer's Disease from structural Magnetic Resonance Imaging (sMRI). We assess the quality of prototypes under a systematic evaluation framework, propose new functionally grounded metrics to evaluate brain prototypes and develop an evaluation scheme to assess their coherency with domain experts. Our results show that PIPNet3D is an interpretable, compact model for Alzheimer's diagnosis with its reasoning well aligned to medical domain knowledge. Notably, PIPNet3D achieves the same accuracy as its blackbox counterpart; and removing the remaining clinically irrelevant prototypes from its decision process does not decrease predictive performance.
Macroscale fracture surface segmentation via semi-supervised learning considering the structural similarity
Venue: arXiv cs.LG 2024
Year: 2024

To this date the safety assessment of materials, used for example in the nuclear power sector, commonly relies on a fracture mechanical analysis utilizing macroscopic concepts, where a global load quantity K or J is compared to the materials fracture toughness curve. Part of the experimental effort involved in these concepts is dedicated to the quantitative analysis of fracture surfaces. Within the scope of this study a methodology for the semi-supervised training of deep learning models for fracture surface segmentation on a macroscopic level was established. Therefore, three distinct and unique datasets were created to analyze the influence of structural similarity on the segmentation capability. The structural similarity differs due to the assessed materials and specimen, as well as imaging-induced variance due to fluctuations in image acquisition in different laboratories. The datasets correspond to typical isolated laboratory conditions, complex real-world circumstances, and a curated subset of the two. We implemented a weak-to-strong consistency regularization for semi-supervised learning. On the heterogeneous dataset we were able to train robust and well-generalizing models that learned feature representations from images across different domains without observing a significant drop in prediction quality. Furthermore, our approach reduced the number of labeled images required for training by a factor of 6. To demonstrate the success of our method and the benefit of our approach for the fracture mechanics assessment, we utilized the models for initial crack size measurements with the area average method. For the laboratory setting, the deep learning assisted measurements proved to have the same quality as manual measurements. For models trained on the heterogeneous dataset, very good measurement accuracies with mean deviations smaller than 1 % could be achieved...
Generating Diverse Agricultural Data for Vision-Based Farming Applications
Venue: arXiv cs.CV 2024
Year: 2024

We present a specialized procedural model for generating synthetic agricultural scenes, focusing on soybean crops, along with various weeds. This model is capable of simulating distinct growth stages of these plants, diverse soil conditions, and randomized field arrangements under varying lighting conditions. The integration of real-world textures and environmental factors into the procedural generation process enhances the photorealism and applicability of the synthetic data. Our dataset includes 12,000 images with semantic labels, offering a comprehensive resource for computer vision tasks in precision agriculture, such as semantic segmentation for autonomous weed control. We validate our model's effectiveness by comparing the synthetic data against real agricultural images, demonstrating its potential to significantly augment training data for machine learning models in agriculture. This approach not only provides a cost-effective solution for generating high-quality, diverse data but also addresses specific needs in agricultural vision tasks that are not fully covered by general-purpose models.
ViTAR: Vision Transformer with Any Resolution
Venue: arXiv cs.CV 2024
Year: 2024

This paper tackles a significant challenge faced by Vision Transformers (ViTs): their constrained scalability across different image resolutions. Typically, ViTs experience a performance decline when processing resolutions different from those seen during training. Our work introduces two key innovations to address this issue. Firstly, we propose a novel module for dynamic resolution adjustment, designed with a single Transformer block, specifically to achieve highly efficient incremental token integration. Secondly, we introduce fuzzy positional encoding in the Vision Transformer to provide consistent positional awareness across multiple resolutions, thereby preventing overfitting to any single training resolution. Our resulting model, ViTAR (Vision Transformer with Any Resolution), demonstrates impressive adaptability, achieving 83.3\% top-1 accuracy at a 1120x1120 resolution and 80.4\% accuracy at a 4032x4032 resolution, all while reducing computational costs. ViTAR also shows strong performance in downstream tasks such as instance and semantic segmentation and can easily combined with self-supervised learning techniques like Masked AutoEncoder. Our work provides a cost-effective solution for enhancing the resolution scalability of ViTs, paving the way for more versatile and efficient high-resolution image processing.
Damage Mechanics Challenge: Predictions based on the phase field fracture model
Venue: arXiv cs.CE 2024
Year: 2024

In this work, we describe our contribution to the Purdue-SANDIA-LLNL \emph{Damage Mechanics Challenge}. The phase field fracture model is adopted to blindly estimate the failure characteristics of the challenge test, an unconventional three-point bending experiment on an additively manufactured rock resembling a type of gypsum. The model is formulated in a variationally consistent fashion, incorporating a volumetric-deviatoric strain energy decomposition, and the numerical implementation adopts a monolithic unconditionally stable solution scheme. Our focus is on providing an efficient and simple yet rigorous approach capable of delivering accurate predictions based solely on physical parameters. Model inputs are Young's modulus $E$, Poisson's ratio $ν$, toughness $G_c$ and strength $σ_c$ (as determined by the choice of phase field length scale $\ell$). We show that a single mode I three-point bending test is sufficient to calibrate the model, and that the calibrated model can then reliably predict the force versus displacement responses, crack paths and surface crack morphologies of more intricate three-point bending experiments that are inherently mixed-mode. Importantly, our peak load, crack trajectory and crack surface morphology predictions for the challenge test, submitted before the experimental data was released, show a remarkable agreement with experiments. The characteristics of the challenge, and how changes in these can impact the predictive abilities of phase field fracture models, are also discussed.
Convexity of near-optimal orthogonal-pair-free sets on the unit sphere
Venue: arXiv cs.CG 2024
Year: 2024

A subset $S$ of the unit sphere $\mathbb{S}^2$ is called orthogonal-pair-free if and only if there do not exist two distinct points $u, v \in S$ at distance $\fracπ{2}$ from each other. Witsenhausen \cite{witsenhausen} asked the following question: {\it What is the least upper bound $α_3$ on the Lesbegue measure of any measurable orthogonal-pair-free subset of $\mathbb{S}^2$?} We prove the following result in this paper: Let $\mathcal{A}$ be the collection of all orthogonal-pair-free sets $S$ such that $S$ consists of a finite number of mutually disjoint convex sets. Then, $α_3 = \limsup_{S \in \mathcal{A}} μ(S)$. Thus, if the double cap conjecture \cite{kalai1} is not true, there is a set in $\mathcal{A}$ with measure strictly greater than the measure of the double cap.
iFace: Hand-Over-Face Gesture Recognition Leveraging Impedance Sensing
Venue: arXiv cs.HC 2024
Year: 2024

Hand-over-face gestures can provide important implicit interactions during conversations, such as frustration or excitement. However, in situations where interlocutors are not visible, such as phone calls or textual communication, the potential meaning contained in the hand-over-face gestures is lost. In this work, we present iFace, an unobtrusive, wearable impedance-sensing solution for recognizing different hand-over-face gestures. In contrast to most existing works, iFace does not require the placement of sensors on the user's face or hands. Instead, we proposed a novel sensing configuration, the shoulders, which remains invisible to both the user and outside observers. The system can monitor the shoulder-to-shoulder impedance variation caused by gestures through electrodes attached to each shoulder. We evaluated iFace in a user study with eight participants, collecting six kinds of hand-over-face gestures with different meanings. Using a convolutional neural network and a user-dependent classification, iFace reaches 82.58 \% macro F1 score. We discuss potential application scenarios of iFace as an implicit interaction interface.
DELTA: Pre-train a Discriminative Encoder for Legal Case Retrieval via Structural Word Alignment
Venue: arXiv cs.IR 2024
Year: 2024

Recent research demonstrates the effectiveness of using pre-trained language models for legal case retrieval. Most of the existing works focus on improving the representation ability for the contextualized embedding of the [CLS] token and calculate relevance using textual semantic similarity. However, in the legal domain, textual semantic similarity does not always imply that the cases are relevant enough. Instead, relevance in legal cases primarily depends on the similarity of key facts that impact the final judgment. Without proper treatments, the discriminative ability of learned representations could be limited since legal cases are lengthy and contain numerous non-key facts. To this end, we introduce DELTA, a discriminative model designed for legal case retrieval. The basic idea involves pinpointing key facts in legal cases and pulling the contextualized embedding of the [CLS] token closer to the key facts while pushing away from the non-key facts, which can warm up the case embedding space in an unsupervised manner. To be specific, this study brings the word alignment mechanism to the contextual masked auto-encoder. First, we leverage shallow decoders to create information bottlenecks, aiming to enhance the representation ability. Second, we employ the deep decoder to enable translation between different structures, with the goal of pinpointing key facts to enhance discriminative ability. Comprehensive experiments conducted on publicly available legal benchmarks show that our approach can outperform existing state-of-the-art methods in legal case retrieval. It provides a new perspective on the in-depth understanding and processing of legal case documents.