OpenRaiser/Intern-Atlas huggingface dataset:
Counteracting Concept Drift by Learning with Future Malware Predictions
Venue: arXiv cs.CR 2024
Year: 2024

The accuracy of deployed malware-detection classifiers degrades over time due to changes in data distributions and increasing discrepancies between training and testing data. This phenomenon is known as the concept drift. While the concept drift can be caused by various reasons in general, new malicious files are created by malware authors with a clear intention of avoiding detection. The existence of the intention opens a possibility for predicting such future samples. Including predicted samples in training data should consequently increase the accuracy of the classifiers on new testing data. We compare two methods for predicting future samples: (1) adversarial training and (2) generative adversarial networks (GANs). The first method explicitly seeks for adversarial examples against the classifier that are then used as a part of training data. Similarly, GANs also generate synthetic training data. We use GANs to learn changes in data distributions within different time periods of training data and then apply these changes to generate samples that could be in testing data. We compare these prediction methods on two different datasets: (1) Ember public dataset and (2) the internal dataset of files incoming to Avast. We show that while adversarial training yields more robust classifiers, this method is not a good predictor of future malware in general. This is in contrast with previously reported positive results in different domains (including natural language processing and spam detection). On the other hand, we show that GANs can be successfully used as predictors of future malware. We specifically examine malware families that exhibit significant changes in their data distributions over time and the experimental results confirm that GAN-based predictions can significantly improve the accuracy of the classifier on new, previously unseen data.
Identification of cardiovascular diseases through ECG classification using wavelet transformation
Venue: arXiv cs.CE 2024
Year: 2024

Cardiovascular diseases are the leading cause of mortality globally, necessitating advancements in diagnostic techniques. This study explores the application of wavelet transformation for classifying electrocardiogram (ECG) signals to identify various cardiovascular conditions. Utilizing the MIT-BIH Arrhythmia Database, we employed both continuous and discrete wavelet transforms to decompose ECG signals into frequency sub-bands, from which we extracted eight statistical features per band. These features were then used to train and test various classifiers, including K-Nearest Neighbors and Support Vector Machines, among others. The classifiers demonstrated high efficacy, with some achieving an accuracy of up to 96% on test data, suggesting that wavelet-based feature extraction significantly enhances the prediction of cardiovascular abnormalities in ECG data. The findings advocate for further exploration of wavelet transforms in medical diagnostics to improve automation and accuracy in disease detection. Future work will focus on optimizing feature selection and classifier parameters to refine predictive performance further.
EQO: Exploring Ultra-Efficient Private Inference with Winograd-Based Protocol and Quantization Co-Optimization
Venue: arXiv cs.CR 2024
Year: 2024

Private convolutional neural network (CNN) inference based on secure two-party computation (2PC) suffers from high communication and latency overhead, especially from convolution layers. In this paper, we propose EQO, a quantized 2PC inference framework that jointly optimizes the CNNs and 2PC protocols. EQO features a novel 2PC protocol that combines Winograd transformation with quantization for efficient convolution computation. However, we observe naively combining quantization and Winograd convolution is sub-optimal: Winograd transformations introduce extensive local additions and weight outliers that increase the quantization bit widths and require frequent bit width conversions with non-negligible communication overhead. Therefore, at the protocol level, we propose a series of optimizations for the 2PC inference graph to minimize the communication. At the network level, We develop a sensitivity-based mixed-precision quantization algorithm to optimize network accuracy given communication constraints. We further propose a 2PC-friendly bit re-weighting algorithm to accommodate weight outliers without increasing bit widths. With extensive experiments, EQO demonstrates 11.7x, 3.6x, and 6.3x communication reduction with 1.29%, 1.16%, and 1.29% higher accuracy compared to state-of-the-art frameworks SiRNN, COINN, and CoPriv, respectively.
Few-shot Name Entity Recognition on StackOverflow
Venue: arXiv cs.CL 2024
Year: 2024

StackOverflow, with its vast question repository and limited labeled examples, raise an annotation challenge for us. We address this gap by proposing RoBERTa+MAML, a few-shot named entity recognition (NER) method leveraging meta-learning. Our approach, evaluated on the StackOverflow NER corpus (27 entity types), achieves a 5% F1 score improvement over the baseline. We improved the results further domain-specific phrase processing enhance results.
DeferredGS: Decoupled and Editable Gaussian Splatting with Deferred Shading
Venue: arXiv cs.CV 2024
Year: 2024

Reconstructing and editing 3D objects and scenes both play crucial roles in computer graphics and computer vision. Neural radiance fields (NeRFs) can achieve realistic reconstruction and editing results but suffer from inefficiency in rendering. Gaussian splatting significantly accelerates rendering by rasterizing Gaussian ellipsoids. However, Gaussian splatting utilizes a single Spherical Harmonic (SH) function to model both texture and lighting, limiting independent editing capabilities of these components. Recently, attempts have been made to decouple texture and lighting with the Gaussian splatting representation but may fail to produce plausible geometry and decomposition results on reflective scenes. Additionally, the forward shading technique they employ introduces noticeable blending artifacts during relighting, as the geometry attributes of Gaussians are optimized under the original illumination and may not be suitable for novel lighting conditions. To address these issues, we introduce DeferredGS, a method for decoupling and editing the Gaussian splatting representation using deferred shading. To achieve successful decoupling, we model the illumination with a learnable environment map and define additional attributes such as texture parameters and normal direction on Gaussians, where the normal is distilled from a jointly trained signed distance function. More importantly, we apply deferred shading, resulting in more realistic relighting effects compared to previous methods. Both qualitative and quantitative experiments demonstrate the superior performance of DeferredGS in novel view synthesis and editing tasks.
A Review on Machine Learning Algorithms for Dust Aerosol Detection using Satellite Data
Venue: arXiv cs.CV 2024
Year: 2024

Dust storms are associated with certain respiratory illnesses across different areas in the world. Researchers have devoted time and resources to study the elements surrounding dust storm phenomena. This paper reviews the efforts of those who have investigated dust aerosols using sensors onboard of satellites using machine learning-based approaches. We have reviewed the most common issues revolving dust aerosol modeling using different datasets and different sensors from a historical perspective. Our findings suggest that multi-spectral approaches based on linear and non-linear combinations of spectral bands are some of the most successful for visualization and quantitative analysis; however, when researchers have leveraged machine learning, performance has been improved and new opportunities to solve unique problems arise.
ViFu: Multiple 360$^\circ$ Objects Reconstruction with Clean Background via Visible Part Fusion
Venue: arXiv cs.CV 2024
Year: 2024

In this paper, we propose a method to segment and recover a static, clean background and multiple 360$^\circ$ objects from observations of scenes at different timestamps. Recent works have used neural radiance fields to model 3D scenes and improved the quality of novel view synthesis, while few studies have focused on modeling the invisible or occluded parts of the training images. These under-reconstruction parts constrain both scene editing and rendering view selection, thereby limiting their utility for synthetic data generation for downstream tasks. Our basic idea is that, by observing the same set of objects in various arrangement, so that parts that are invisible in one scene may become visible in others. By fusing the visible parts from each scene, occlusion-free rendering of both background and foreground objects can be achieved. We decompose the multi-scene fusion task into two main components: (1) objects/background segmentation and alignment, where we leverage point cloud-based methods tailored to our novel problem formulation; (2) radiance fields fusion, where we introduce visibility field to quantify the visible information of radiance fields, and propose visibility-aware rendering for the fusion of series of scenes, ultimately obtaining clean background and 360$^\circ$ object rendering. Comprehensive experiments were conducted on synthetic and real datasets, and the results demonstrate the effectiveness of our method.
Exploring Text-to-Motion Generation with Human Preference
Venue: arXiv cs.LG 2024
Year: 2024

This paper presents an exploration of preference learning in text-to-motion generation. We find that current improvements in text-to-motion generation still rely on datasets requiring expert labelers with motion capture systems. Instead, learning from human preference data does not require motion capture systems; a labeler with no expertise simply compares two generated motions. This is particularly efficient because evaluating the model's output is easier than gathering the motion that performs a desired task (e.g. backflip). To pioneer the exploration of this paradigm, we annotate 3,528 preference pairs generated by MotionGPT, marking the first effort to investigate various algorithms for learning from preference data. In particular, our exploration highlights important design choices when using preference data. Additionally, our experimental results show that preference learning has the potential to greatly improve current text-to-motion generative models. Our code and dataset are publicly available at https://github.com/THU-LYJ-Lab/InstructMotion}{https://github.com/THU-LYJ-Lab/InstructMotion to further facilitate research in this area.
Crooked indifferentiability of the Feistel Construction
Venue: arXiv cs.CR 2024
Year: 2024

The Feistel construction is a fundamental technique for building pseudorandom permutations and block ciphers. This paper shows that a simple adaptation of the construction is resistant, even to algorithm substitution attacks -- that is, adversarial subversion -- of the component round functions. Specifically, we establish that a Feistel-based construction with more than $2000n/\log(1/ε)$ rounds can transform a subverted random function -- which disagrees with the original one at a small fraction (denoted by $ε$) of inputs -- into an object that is \emph{crooked-indifferentiable} from a random permutation, even if the adversary is aware of all the randomness used in the transformation. We also provide a lower bound showing that the construction cannot use fewer than $2n/\log(1/ε)$ rounds to achieve crooked-indifferentiable security.
CompGS: Efficient 3D Scene Representation via Compressed Gaussian Splatting
Venue: arXiv cs.CV 2024
Year: 2024

Gaussian splatting, renowned for its exceptional rendering quality and efficiency, has emerged as a prominent technique in 3D scene representation. However, the substantial data volume of Gaussian splatting impedes its practical utility in real-world applications. Herein, we propose an efficient 3D scene representation, named Compressed Gaussian Splatting (CompGS), which harnesses compact Gaussian primitives for faithful 3D scene modeling with a remarkably reduced data size. To ensure the compactness of Gaussian primitives, we devise a hybrid primitive structure that captures predictive relationships between each other. Then, we exploit a small set of anchor primitives for prediction, allowing the majority of primitives to be encapsulated into highly compact residual forms. Moreover, we develop a rate-constrained optimization scheme to eliminate redundancies within such hybrid primitives, steering our CompGS towards an optimal trade-off between bitrate consumption and representation efficacy. Experimental results show that the proposed CompGS significantly outperforms existing methods, achieving superior compactness in 3D scene representation without compromising model accuracy and rendering quality. Our code will be released on GitHub for further research.
Scoring Time Intervals using Non-Hierarchical Transformer For Automatic Piano Transcription
Venue: arXiv cs.SD 2024
Year: 2024

The neural semi-Markov Conditional Random Field (semi-CRF) framework has demonstrated promise for event-based piano transcription. In this framework, all events (notes or pedals) are represented as closed time intervals tied to specific event types. The neural semi-CRF approach requires an interval scoring matrix that assigns a score for every candidate interval. However, designing an efficient and expressive architecture for scoring intervals is not trivial. This paper introduces a simple method for scoring intervals using scaled inner product operations that resemble how attention scoring is done in transformers. We show theoretically that, due to the special structure from encoding the non-overlapping intervals, under a mild condition, the inner product operations are expressive enough to represent an ideal scoring matrix that can yield the correct transcription result. We then demonstrate that an encoder-only structured non-hierarchical transformer backbone, operating only on a low-time-resolution feature map, is capable of transcribing piano notes and pedals with high accuracy and time precision. The experiment shows that our approach achieves the new state-of-the-art performance across all subtasks in terms of the F1 measure on the Maestro dataset.
Tokenization, Fusion, and Augmentation: Towards Fine-grained Multi-modal Entity Representation
Venue: arXiv cs.AI 2024
Year: 2024

Multi-modal knowledge graph completion (MMKGC) aims to discover unobserved knowledge from given knowledge graphs, collaboratively leveraging structural information from the triples and multi-modal information of the entities to overcome the inherent incompleteness. Existing MMKGC methods usually extract multi-modal features with pre-trained models, resulting in coarse handling of multi-modal entity information, overlooking the nuanced, fine-grained semantic details and their complex interactions. To tackle this shortfall, we introduce a novel framework MyGO to tokenize, fuse, and augment the fine-grained multi-modal representations of entities and enhance the MMKGC performance. Motivated by the tokenization technology, MyGO tokenizes multi-modal entity information as fine-grained discrete tokens and learns entity representations with a cross-modal entity encoder. To further augment the multi-modal representations, MyGO incorporates fine-grained contrastive learning to highlight the specificity of the entity representations. Experiments on standard MMKGC benchmarks reveal that our method surpasses 19 of the latest models, underlining its superior performance. Code and data can be found in https://github.com/zjukg/MyGO
Virtually Enriched NYU Depth V2 Dataset for Monocular Depth Estimation: Do We Need Artificial Augmentation?
Venue: arXiv cs.CV 2024
Year: 2024

We present ANYU, a new virtually augmented version of the NYU depth v2 dataset, designed for monocular depth estimation. In contrast to the well-known approach where full 3D scenes of a virtual world are utilized to generate artificial datasets, ANYU was created by incorporating RGB-D representations of virtual reality objects into the original NYU depth v2 images. We specifically did not match each generated virtual object with an appropriate texture and a suitable location within the real-world image. Instead, an assignment of texture, location, lighting, and other rendering parameters was randomized to maximize a diversity of the training data, and to show that it is randomness that can improve the generalizing ability of a dataset. By conducting extensive experiments with our virtually modified dataset and validating on the original NYU depth v2 and iBims-1 benchmarks, we show that ANYU improves the monocular depth estimation performance and generalization of deep neural networks with considerably different architectures, especially for the current state-of-the-art VPD model. To the best of our knowledge, this is the first work that augments a real-world dataset with randomly generated virtual 3D objects for monocular depth estimation. We make our ANYU dataset publicly available in two training configurations with 10% and 100% additional synthetically enriched RGB-D pairs of training images, respectively, for efficient training and empirical exploration of virtual augmentation at https://github.com/ABrain-One/ANYU
Q2A: Querying Implicit Fully Continuous Feature Pyramid to Align Features for Medical Image Segmentation
Venue: arXiv cs.CV 2024
Year: 2024

Recent medical image segmentation methods apply implicit neural representation (INR) to the decoder for achieving a continuous coordinate decoding to tackle the drawback of conventional discrete grid-based data representations. However, the INR-based decoder cannot well handle the feature misalignment problem brought about by the naive latent code acquisition strategy in INR. Although there exist many feature alignment works, they all adopt a progressive multi-step aligning paradigm on a discrete feature pyramid, which is incompatible with the continuous one-step characteristics of INR-based decoder, and thus fails to be the solution. Therefore, we propose Q2A, a novel one-step query-based aligning paradigm, to solve the feature misalignment problem in the INR-based decoder. Specifically, for each target coordinate, Q2A first generates several queries depicting the spatial offsets and the cell resolutions of the contextual features aligned to the coordinate, then calculates the corresponding aligned features by feeding the queries into a novel implicit fully continuous feature pyramid (FCFP), finally fuses the aligned features to predict the class distribution. In FCFP, we further propose a novel universal partition-and-aggregate strategy (P&A) to replace the naive interpolation strategy for latent code acquisition in INR, which mitigates the information loss problem that occurs when the query cell resolution is relatively large and achieves an effective feature decoding at arbitrary continuous resolution. We conduct extensive experiments on two medical datasets, i.e. Glas and Synapse, and a universal dataset, i.e. Cityscapes, and they show the superiority of the proposed Q2A.
Computable domains of a Halting Function
Venue: arXiv cs.LO 2024
Year: 2024

We discuss the possibility of constructing a function that validates the definition or not definition of the partial recursive functions of one variable. This is a topic in computability theory, which was first approached by Alan M. Turing in 1936 in his foundational work "On Computable Numbers". Here we face it using the Model of computability of the recursive functions instead of the Turing's machines, but the results are transferable from one to another paradigm with ease. Recursive functions that are not defined at a given point, correspond to the Turing machines that "do not end" for a given input. What we propose Is a slight slip from the orthodox point of view: the issue of the self-reference and of the self-validation is not an impediment in imperative languages.
MMCode: Benchmarking Multimodal Large Language Models for Code Generation with Visually Rich Programming Problems
Venue: arXiv cs.CL 2024
Year: 2024

Programming often involves converting detailed and complex specifications into code, a process during which developers typically utilize visual aids to more effectively convey concepts. While recent developments in Large Multimodal Models have demonstrated remarkable abilities in visual reasoning and mathematical tasks, there is little work on investigating whether these models can effectively interpret visual elements for code generation. To this end, we present MMCode, the first multi-modal coding dataset for evaluating algorithmic problem-solving skills in visually rich contexts. MMCode contains 3,548 questions and 6,620 images collected from real-world programming challenges harvested from 10 code competition websites, presenting significant challenges due to the extreme demand for reasoning abilities. Our experiment results show that current state-of-the-art models struggle to solve these problems. The results highlight the lack of powerful vision-code models, and we hope MMCode can serve as an inspiration for future works in this domain. The data and code are publicly available at https://github.com/likaixin2000/MMCode.
Performance analysis of satellite-terrestrial integrated radio access networks based on stochastic geometry
Venue: arXiv cs.IT 2024
Year: 2024

To enhance coverage and improve service continuity, satellite-terrestrial integrated radio access network (STIRAN) has been seen as an essential trend in the development of 6G. However, there is still a lack of theoretical analysis on its coverage performance. To fill this gap, we first establish a system model to characterize a typical scenario where low-earth-orbit (LEO) satellites and terrestrial base stations are both deployed. Then, stochastic geometry is utilized to analyze the downlink coverage probability under the setting of shared frequency and distinct frequencies. Specifically, we derive mathematical expressions for the distances distribution from the serving station to the typical user and the associated probability based on the maximum bias power selection strategy (Max-BPR). Taking into account real-world satellite antenna beamforming patterns in two system scenarios, we derive the downlink coverage probabilities in terms of parameters such as base station density and orbital inclination. Finally, the correctness of the theoretical derivations is verified through experimental simulations, and the influence of network design parameters on the downlink coverage probability is analyzed.
Fuse after Align: Improving Face-Voice Association Learning via Multimodal Encoder
Venue: arXiv cs.CV 2024
Year: 2024

Today, there have been many achievements in learning the association between voice and face. However, most previous work models rely on cosine similarity or L2 distance to evaluate the likeness of voices and faces following contrastive learning, subsequently applied to retrieval and matching tasks. This method only considers the embeddings as high-dimensional vectors, utilizing a minimal scope of available information. This paper introduces a novel framework within an unsupervised setting for learning voice-face associations. By employing a multimodal encoder after contrastive learning and addressing the problem through binary classification, we can learn the implicit information within the embeddings in a more effective and varied manner. Furthermore, by introducing an effective pair selection method, we enhance the learning outcomes of both contrastive learning and the matching task. Empirical evidence demonstrates that our framework achieves state-of-the-art results in voice-face matching, verification, and retrieval tasks, improving verification by approximately 3%, matching by about 2.5%, and retrieval by around 1.3%.