OpenRaiser/Intern-Atlas huggingface dataset:
1-800-SHARED-TASKS @ NLU of Devanagari Script Languages: Detection of Language, Hate Speech, and Targets using LLMs
Venue: arXiv cs.CL 2024
Year: 2024

This paper presents a detailed system description of our entry for the CHiPSAL 2025 shared task, focusing on language detection, hate speech identification, and target detection in Devanagari script languages. We experimented with a combination of large language models and their ensembles, including MuRIL, IndicBERT, and Gemma-2, and leveraged unique techniques like focal loss to address challenges in the natural understanding of Devanagari languages, such as multilingual processing and class imbalance. Our approach achieved competitive results across all tasks: F1 of 0.9980, 0.7652, and 0.6804 for Sub-tasks A, B, and C respectively. This work provides insights into the effectiveness of transformer models in tasks with domain-specific and linguistic challenges, as well as areas for potential improvement in future iterations.
Enhancing Phishing Detection through Feature Importance Analysis and Explainable AI: A Comparative Study of CatBoost, XGBoost, and EBM Models
Venue: arXiv cs.CR 2024
Year: 2024

Phishing attacks remain a persistent threat to online security, demanding robust detection methods. This study investigates the use of machine learning to identify phishing URLs, emphasizing the crucial role of feature selection and model interpretability for improved performance. Employing Recursive Feature Elimination, the research pinpointed key features like "length_url," "time_domain_activation" and "Page_rank" as strong indicators of phishing attempts. The study evaluated various algorithms, including CatBoost, XGBoost, and Explainable Boosting Machine, assessing their robustness and scalability. XGBoost emerged as highly efficient in terms of runtime, making it well-suited for large datasets. CatBoost, on the other hand, demonstrated resilience by maintaining high accuracy even with reduced features. To enhance transparency and trustworthiness, Explainable AI techniques, such as SHAP, were employed to provide insights into feature importance. The study's findings highlight that effective feature selection and model interpretability can significantly bolster phishing detection systems, paving the way for more efficient and adaptable defenses against evolving cyber threats
Computable Model-Independent Bounds for Adversarial Quantum Machine Learning
Venue: arXiv cs.LG 2024
Year: 2024

By leveraging the principles of quantum mechanics, QML opens doors to novel approaches in machine learning and offers potential speedup. However, machine learning models are well-documented to be vulnerable to malicious manipulations, and this susceptibility extends to the models of QML. This situation necessitates a thorough understanding of QML's resilience against adversarial attacks, particularly in an era where quantum computing capabilities are expanding. In this regard, this paper examines model-independent bounds on adversarial performance for QML. To the best of our knowledge, we introduce the first computation of an approximate lower bound for adversarial error when evaluating model resilience against sophisticated quantum-based adversarial attacks. Experimental results are compared to the computed bound, demonstrating the potential of QML models to achieve high robustness. In the best case, the experimental error is only 10% above the estimated bound, offering evidence of the inherent robustness of quantum models. This work not only advances our theoretical understanding of quantum model resilience but also provides a precise reference bound for the future development of robust QML algorithms.
Case Frames and Case-Based Arguments in Statutory Interpretation
Venue: arXiv cs.SC 2024
Year: 2024

We introduce a novel conceptual Case Frame model that represents the content of cases involving statutory interpretation within civil law frameworks, accompanied by an associated argument scheme enriched with critical questions. By validating our approach with a modest dataset, we demonstrate its robustness and practical applicability. Our model not only provides a structured method for analyzing statutory interpretation but also highlights the distinct needs of lawyers operating under statutory law compared to those reasoning with common law precedents. The model presented here is a step towards developing a hybrid Machine Learning and Argumentation system that includes a module for constructing well-structured arguments from textual datasets.
GraphRPM: Risk Pattern Mining on Industrial Large Attributed Graphs
Venue: arXiv cs.LG 2024
Year: 2024

Graph-based patterns are extensively employed and favored by practitioners within industrial companies due to their capacity to represent the behavioral attributes and topological relationships among users, thereby offering enhanced interpretability in comparison to black-box models commonly utilized for classification and recognition tasks. For instance, within the scenario of transaction risk management, a graph pattern that is characteristic of a particular risk category can be readily employed to discern transactions fraught with risk, delineate networks of criminal activity, or investigate the methodologies employed by fraudsters. Nonetheless, graph data in industrial settings is often characterized by its massive scale, encompassing data sets with millions or even billions of nodes, making the manual extraction of graph patterns not only labor-intensive but also necessitating specialized knowledge in particular domains of risk. Moreover, existing methodologies for mining graph patterns encounter significant obstacles when tasked with analyzing large-scale attributed graphs. In this work, we introduce GraphRPM, an industry-purpose parallel and distributed risk pattern mining framework on large attributed graphs. The framework incorporates a novel edge-involved graph isomorphism network alongside optimized operations for parallel graph computation, which collectively contribute to a considerable reduction in computational complexity and resource expenditure. Moreover, the intelligent filtration of efficacious risky graph patterns is facilitated by the proposed evaluation metrics. Comprehensive experimental evaluations conducted on real-world datasets of varying sizes substantiate the capability of GraphRPM to adeptly address the challenges inherent in mining patterns from large-scale industrial attributed graphs, thereby underscoring its substantial value for industrial deployment.
Multi-scale Frequency Enhancement Network for Blind Image Deblurring
Venue: arXiv cs.CV 2024
Year: 2024

Image deblurring is an essential image preprocessing technique, aiming to recover clear and detailed images form blurry ones. However, existing algorithms often fail to effectively integrate multi-scale feature extraction with frequency enhancement, limiting their ability to reconstruct fine textures. Additionally, non-uniform blur in images also restricts the effectiveness of image restoration. To address these issues, we propose a multi-scale frequency enhancement network (MFENet) for blind image deblurring. To capture the multi-scale spatial and channel information of blurred images, we introduce a multi-scale feature extraction module (MS-FE) based on depthwise separable convolutions, which provides rich target features for deblurring. We propose a frequency enhanced blur perception module (FEBP) that employs wavelet transforms to extract high-frequency details and utilizes multi-strip pooling to perceive non-uniform blur, combining multi-scale information with frequency enhancement to improve the restoration of image texture details. Experimental results on the GoPro and HIDE datasets demonstrate that the proposed method achieves superior deblurring performance in both visual quality and objective evaluation metrics. Furthermore, in downstream object detection tasks, the proposed blind image deblurring algorithm significantly improves detection accuracy, further validating its effectiveness androbustness in the field of image deblurring.
Maximizing Nash Social Welfare in 2-Value Instances: A Simpler Proof for the Half-Integer Case
Venue: arXiv cs.GT 2024
Year: 2024

A set of $m$ indivisible goods is to be allocated to a set of $n$ agents. Each agent $i$ has an additive valuation function $v_i$ over goods. The value of a good $g$ for agent $i$ is either $1$ or $s$, where $s$ is a fixed rational number greater than one, and the value of a bundle of goods is the sum of the values of the goods in the bundle. An \emph{allocation} $X$ is a partition of the goods into bundles $X_1$, \ldots, $X_n$, one for each agent. The \emph{Nash Social Welfare} ($\NSW$) of an allocation $X$ is defined as \[ \NSW(X) = \left( \prod_i v_i(X_i) \right)^{\sfrac{1}{n}}.\] The \emph{$\NSW$-allocation} maximizes the Nash Social Welfare. In~\cite{NSW-twovalues-halfinteger} it was shown that the $\NSW$-allocation can be computed in polynomial time, if $s$ is an integer or a half-integer, and that the problem is NP-complete otherwise. The proof for the half-integer case is quite involved. In this note we give a simpler and shorter proof
Mamba-based Decoder-Only Approach with Bidirectional Speech Modeling for Speech Recognition
Venue: arXiv cs.SD 2024
Year: 2024

Selective state space models (SSMs) represented by Mamba have demonstrated their computational efficiency and promising outcomes in various tasks, including automatic speech recognition (ASR). Mamba has been applied to ASR task with the attention-based encoder-decoder framework, where the cross-attention mechanism between encoder and decoder remains. This paper explores the capability of Mamba as the decoder-only architecture in ASR task. Our MAmba-based DEcoder-ONly approach (MADEON) consists of a single decoder that takes speech tokens as a condition and predicts text tokens in an autoregressive manner. To enhance MADEON, we further propose speech prefixing that performs bidirectional processing on speech tokens, which enriches the contextual information in the hidden states. Our experiments show that MADEON significantly outperforms a non-selective SSM. The combination of speech prefixing and the recently proposed Mamba-2 yields comparable performance to Transformer-based models on large datasets.
MapSAM: Adapting Segment Anything Model for Automated Feature Detection in Historical Maps
Venue: arXiv cs.CV 2024
Year: 2024

Automated feature detection in historical maps can significantly accelerate the reconstruction of the geospatial past. However, this process is often constrained by the time-consuming task of manually digitizing sufficient high-quality training data. The emergence of visual foundation models, such as the Segment Anything Model (SAM), offers a promising solution due to their remarkable generalization capabilities and rapid adaptation to new data distributions. Despite this, directly applying SAM in a zero-shot manner to historical map segmentation poses significant challenges, including poor recognition of certain geospatial features and a reliance on input prompts, which limits its ability to be fully automated. To address these challenges, we introduce MapSAM, a parameter-efficient fine-tuning strategy that adapts SAM into a prompt-free and versatile solution for various downstream historical map segmentation tasks. Specifically, we employ Weight-Decomposed Low-Rank Adaptation (DoRA) to integrate domain-specific knowledge into the image encoder. Additionally, we develop an automatic prompt generation process, eliminating the need for manual input. We further enhance the positional prompt in SAM, transforming it into a higher-level positional-semantic prompt, and modify the cross-attention mechanism in the mask decoder with masked attention for more effective feature aggregation. The proposed MapSAM framework demonstrates promising performance across two distinct historical map segmentation tasks: one focused on linear features and the other on areal features. Experimental results show that it adapts well to various features, even when fine-tuned with extremely limited data (e.g. 10 shots).
xNVMe: Unleashing Storage Hardware-Software Co-design
Venue: arXiv cs.OS 2024
Year: 2024

NVMe SSD hardware has witnessed widespread deployment as commodity and enterprise hardware due to its high performance and rich feature set. Despite the open specifications of various NVMe protocols by the NVMe Express group and NVMe being of software abstractions to program the underlying hardware. The myriad storage I/O paths such as POSIX storage API, ad-hoc OS mechanisms, and userspace I/O libraries have different syntax and semantics that complicate software development and stand in the way of mass adoption and evolution of the NVMe ecosystem. To unify the diverse I/O storage paths, we built xNVMe that exposes a single message-passing API to support both asynchronous and synchronous communication with NVMe devices. xNVMe provides various command sets to support diverse storage I/O paths in different OS (e.g., Linux, FreeBSD, Windows, and MacOS) and userspace libraries (e.g., SPDK) with minimal overhead. xNVMe is an Open Source project and has gained traction amongst various industry stakeholders. In this paper, we elaborate on the lessons that we have learned in the project during its evolution. We also provide some ongoing and future work planned for the project. We hope the database and storage systems community can join in the effort to both extend xNVMe and leverage it as a building block for innovative co-design of storage systems on modern NVMe hardware.
SIESEF-FusionNet: Spatial Inter-correlation Enhancement and Spatially-Embedded Feature Fusion Network for LiDAR Point Cloud Semantic Segmentation
Venue: arXiv cs.CV 2024
Year: 2024

The ambiguity at the boundaries of different semantic classes in point cloud semantic segmentation often leads to incorrect decisions in intelligent perception systems, such as autonomous driving. Hence, accurate delineation of the boundaries is crucial for improving safety in autonomous driving. A novel spatial inter-correlation enhancement and spatially-embedded feature fusion network (SIESEF-FusionNet) is proposed in this paper, enhancing spatial inter-correlation by combining inverse distance weighting and angular compensation to extract more beneficial spatial information without causing redundancy. Meanwhile, a new spatial adaptive pooling module is also designed, embedding enhanced spatial information into semantic features for strengthening the context-awareness of semantic features. Experimental results demonstrate that 83.7% mIoU and 97.8% OA are achieved by SIESEF-FusionNet on the Toronto3D dataset, with performance superior to other baseline methods. A value of 61.1% mIoU is reached on the semanticKITTI dataset, where a marked improvement in segmentation performance is observed. In addition, the effectiveness and plug-and-play capability of the proposed modules are further verified through ablation studies.
What Do Developers Discuss in Their Workplace? An Analysis of Workplace StackExchange Discussions
Venue: arXiv cs.SE 2024
Year: 2024

Software workplaces are increasingly recognized as key spaces for professional development, where developers encounter various challenges in their roles, which they often discuss in online forums. This paper analyzes 47,368 posts on the Workplace StackExchange site, aggregating developer insights and applying topic modeling techniques. Through manual analysis, we identified 46 distinct topics grouped into seven categories: Employee Wellness, Communication, Career Movement \& Hiring, Conflicts \& Mistakes, Corporate Policies, Management/Supervisor Responsibilities, and Learning \& Technical Skills. Our findings show that approximately 30\% of discussions involve workplace conflicts, marking this as the most prominent topic. Additionally, we found that workplace culture, harassment, and other corporate policy-related issues represent significant areas of difficulty commonly discussed among developers.
On Active Privacy Auditing in Supervised Fine-tuning for White-Box Language Models
Venue: arXiv cs.CL 2024
Year: 2024

The pretraining and fine-tuning approach has become the leading technique for various NLP applications. However, recent studies reveal that fine-tuning data, due to their sensitive nature, domain-specific characteristics, and identifiability, pose significant privacy concerns. To help develop more privacy-resilient fine-tuning models, we introduce a novel active privacy auditing framework, dubbed Parsing, designed to identify and quantify privacy leakage risks during the supervised fine-tuning (SFT) of language models (LMs). The framework leverages improved white-box membership inference attacks (MIAs) as the core technology, utilizing novel learning objectives and a two-stage pipeline to monitor the privacy of the LMs' fine-tuning process, maximizing the exposure of privacy risks. Additionally, we have improved the effectiveness of MIAs on large LMs including GPT-2, Llama2, and certain variants of them. Our research aims to provide the SFT community of LMs with a reliable, ready-to-use privacy auditing tool, and to offer valuable insights into safeguarding privacy during the fine-tuning process. Experimental results confirm the framework's efficiency across various models and tasks, emphasizing notable privacy concerns in the fine-tuning process. Project code available for https://anonymous.4open.science/r/PARSING-4817/.
Edify Image: High-Quality Image Generation with Pixel Space Laplacian Diffusion Models
Venue: arXiv cs.CV 2024
Year: 2024

We introduce Edify Image, a family of diffusion models capable of generating photorealistic image content with pixel-perfect accuracy. Edify Image utilizes cascaded pixel-space diffusion models trained using a novel Laplacian diffusion process, in which image signals at different frequency bands are attenuated at varying rates. Edify Image supports a wide range of applications, including text-to-image synthesis, 4K upsampling, ControlNets, 360 HDR panorama generation, and finetuning for image customization.
Chinese SimpleQA: A Chinese Factuality Evaluation for Large Language Models
Venue: arXiv cs.CL 2024
Year: 2024

New LLM evaluation benchmarks are important to align with the rapid development of Large Language Models (LLMs). In this work, we present Chinese SimpleQA, the first comprehensive Chinese benchmark to evaluate the factuality ability of language models to answer short questions, and Chinese SimpleQA mainly has five properties (i.e., Chinese, Diverse, High-quality, Static, Easy-to-evaluate). Specifically, first, we focus on the Chinese language over 6 major topics with 99 diverse subtopics. Second, we conduct a comprehensive quality control process to achieve high-quality questions and answers, where the reference answers are static and cannot be changed over time. Third, following SimpleQA, the questions and answers are very short, and the grading process is easy-to-evaluate based on OpenAI API. Based on Chinese SimpleQA, we perform a comprehensive evaluation on the factuality abilities of existing LLMs. Finally, we hope that Chinese SimpleQA could guide the developers to better understand the Chinese factuality abilities of their models and facilitate the growth of foundation models.
Enhancing Predictive Maintenance in Mining Mobile Machinery through a TinyML-enabled Hierarchical Inference Network
Venue: arXiv cs.LG 2024
Year: 2024

Mining machinery operating in variable environments faces high wear and unpredictable stress, challenging Predictive Maintenance (PdM). This paper introduces the Edge Sensor Network for Predictive Maintenance (ESN-PdM), a hierarchical inference framework across edge devices, gateways, and cloud services for real-time condition monitoring. The system dynamically adjusts inference locations--on-device, on-gateway, or on-cloud--based on trade-offs among accuracy, latency, and battery life, leveraging Tiny Machine Learning (TinyML) techniques for model optimization on resource-constrained devices. Performance evaluations showed that on-sensor and on-gateway inference modes achieved over 90\% classification accuracy, while cloud-based inference reached 99\%. On-sensor inference reduced power consumption by approximately 44\%, enabling up to 104 hours of operation. Latency was lowest for on-device inference (3.33 ms), increasing when offloading to the gateway (146.67 ms) or cloud (641.71 ms). The ESN-PdM framework provides a scalable, adaptive solution for reliable anomaly detection and PdM, crucial for maintaining machinery uptime in remote environments. By balancing accuracy, latency, and energy consumption, this approach advances PdM frameworks for industrial applications.
Biohybrid Microrobots Based on Jellyfish Stinging Capsules and Janus Particles for In Vitro Deep-Tissue Drug Penetration
Venue: arXiv cs.RO 2024
Year: 2024

Microrobots engineered from self-propelling active particles, extend the reach of robotic operations to submillimeter dimensions and are becoming increasingly relevant for various tasks, such as manipulation of micro/nanoscale cargo, particularly targeted drug delivery. However, achieving deep-tissue penetration and drug delivery remain a challenge. This work developed a novel biohybrid microrobot consisting of jellyfish stinging capsules, which act as natural nanoinjectors for efficient penetration and delivery, assembled onto an active Janus particle (JP). While microrobot transport and navigation was externally controlled by magnetic field-induced rolling, capsule loading onto the JP surface was controlled by electric field. Following precise navigation of the biohybrid microrobots to the vicinity of target tissues, the capsules were activated by a specific enzyme introduced to the solution, which then triggered tubule ejection and release of the preloaded molecules. Use of such microrobots for penetration of and delivery of the preloaded drug/toxin to targeted cancer spheroids and live Caenorhabditis elegans was demonstrated in-vitro. The findings offer insights for future development of bio-inspired microrobots capable of deep penetration and drug delivery. Future directions may involve encapsulation of various drugs within different capsule types for enhanced versatility. This study may also inspire in-vivo applications involving deep tissue drug delivery.
TreeCoders: Trees of Transformers
Venue: arXiv cs.CL 2024
Year: 2024

In this paper, we introduce TreeCoders, a novel family of transformer trees. We moved away from traditional linear transformers to complete k-ary trees. Transformer blocks serve as nodes, and generic classifiers learn to select the best child and route the sequence of tokens to a specific leaf. The selectors, moved outside the transformer blocks, allow for the use of a variety of architecture without further modifications. Furthermore, our proposed architecture supports sparse node activation due to the logarithmic complexity of a tree search. We validate our idea by testing a series of decoder-only tree transformers, achieving competitive results across a diverse range of language datasets. Our study demonstrates that the proposed tree transformer model outperforms a size-equivalent linear transformer model 76\% of the time over a wide range of tree architectures. Furthermore, our proposed model naturally lends itself to distributed implementation.