OpenRaiser/Intern-Atlas huggingface dataset:
Supervised Representation Learning towards Generalizable Assembly State Recognition
Venue: arXiv cs.CV 2024
Year: 2024

Assembly state recognition facilitates the execution of assembly procedures, offering feedback to enhance efficiency and minimize errors. However, recognizing assembly states poses challenges in scalability, since parts are frequently updated, and the robustness to execution errors remains underexplored. To address these challenges, this paper proposes an approach based on representation learning and the novel intermediate-state informed loss function modification (ISIL). ISIL leverages unlabeled transitions between states and demonstrates significant improvements in clustering and classification performance for all tested architectures and losses. Despite being trained exclusively on images without execution errors, thorough analysis on error states demonstrates that our approach accurately distinguishes between correct states and states with various types of execution errors. The integration of the proposed algorithm can offer meaningful assistance to workers and mitigate unexpected losses due to procedural mishaps in industrial settings. The code is available at: https://timschoonbeek.github.io/state_rec
Leveraging Large Language Models for Enhancing the Understandability of Generated Unit Tests
Venue: arXiv cs.SE 2024
Year: 2024

Automated unit test generators, particularly search-based software testing tools like EvoSuite, are capable of generating tests with high coverage. Although these generators alleviate the burden of writing unit tests, they often pose challenges for software engineers in terms of understanding the generated tests. To address this, we introduce UTGen, which combines search-based software testing and large language models to enhance the understandability of automatically generated test cases. We achieve this enhancement through contextualizing test data, improving identifier naming, and adding descriptive comments. Through a controlled experiment with 32 participants from both academia and industry, we investigate how the understandability of unit tests affects a software engineer's ability to perform bug-fixing tasks. We selected bug-fixing to simulate a real-world scenario that emphasizes the importance of understandable test cases. We observe that participants working on assignments with UTGen test cases fix up to 33% more bugs and use up to 20% less time when compared to baseline test cases. From the post-test questionnaire, we gathered that participants found that enhanced test names, test data, and variable names improved their bug-fixing process.
Enhancing Cross-Modal Medical Image Segmentation through Compositionality
Venue: arXiv cs.CV 2024
Year: 2024

Cross-modal medical image segmentation presents a significant challenge, as different imaging modalities produce images with varying resolutions, contrasts, and appearances of anatomical structures. We introduce compositionality as an inductive bias in a cross-modal segmentation network to improve segmentation performance and interpretability while reducing complexity. The proposed network is an end-to-end cross-modal segmentation framework that enforces compositionality on the learned representations using learnable von Mises-Fisher kernels. These kernels facilitate content-style disentanglement in the learned representations, resulting in compositional content representations that are inherently interpretable and effectively disentangle different anatomical structures. The experimental results demonstrate enhanced segmentation performance and reduced computational costs on multiple medical datasets. Additionally, we demonstrate the interpretability of the learned compositional features. Code and checkpoints will be publicly available at: https://github.com/Trustworthy-AI-UU-NKI/Cross-Modal-Segmentation.
MambaCSR: Dual-Interleaved Scanning for Compressed Image Super-Resolution With SSMs
Venue: arXiv cs.CV 2024
Year: 2024

We present MambaCSR, a simple but effective framework based on Mamba for the challenging compressed image super-resolution (CSR) task. Particularly, the scanning strategies of Mamba are crucial for effective contextual knowledge modeling in the restoration process despite it relying on selective state space modeling for all tokens. In this work, we propose an efficient dual-interleaved scanning paradigm (DIS) for CSR, which is composed of two scanning strategies: (i) hierarchical interleaved scanning is designed to comprehensively capture and utilize the most potential contextual information within an image by simultaneously taking advantage of the local window-based and sequential scanning methods; (ii) horizontal-to-vertical interleaved scanning is proposed to reduce the computational cost by leaving the redundancy between the scanning of different directions. To overcome the non-uniform compression artifacts, we also propose position-aligned cross-scale scanning to model multi-scale contextual information. Experimental results on multiple benchmarks have shown the great performance of our MambaCSR in the compressed image super-resolution task. The code will be soon available in~\textcolor{magenta}{\url{https://github.com/renyulin-f/MambaCSR}}.
Esports Training in StarCraft II: Stance Stability and Grip Strength
Venue: arXiv cs.HC 2024
Year: 2024

Esports are a mostly sedentary activity. There is a growing need for investigation into how biomechanical and physical abilities can be optimized for esports through training. One such research avenue concerns the ability of esports players to perform balance tasks due to the prolonged sedentary states that are required to reach the top echelon of performance. Our aim for this work is to describe and compare physical abilities (balance, grip strength, and self-reported training habits) of top Polish StarCraft~2 tournament players. Esports players differed significantly from the reference group in their ability to balance on one leg. Additionally, in a grip strength test, the esports group fared worse than the reference group in all consecutive attempts. Despite self-reported physical activity in the esports group, player fitness requires further research. Training optimization could offset the issues arising from sedentary activity, and intensifying esports training so it could take less time overall.
Decoding Pedestrian Stress on Urban Streets using Electrodermal Activity Monitoring in Virtual Immersive Reality
Venue: arXiv cs.CY 2024
Year: 2024

The pedestrian stress level is shown to significantly influence human cognitive processes and, subsequently, decision-making, e.g., the decision to select a gap and cross a street. This paper systematically studies the stress experienced by a pedestrian when crossing a street under different experimental manipulations by monitoring the ElectroDermal Activity (EDA) using the Galvanic Skin Response (GSR) sensor. To fulfil the research objectives, a dynamic and immersive virtual reality (VR) platform was used, which is suitable for eliciting and capturing pedestrian's emotional responses in conjunction with monitoring their EDA. A total of 171 individuals participated in the experiment, tasked to cross a two-way street at mid-block with no signal control. Mixed effects models were employed to compare the influence of socio-demographics, social influence, vehicle technology, environment, road design, and traffic variables on the stress levels of the participants. The results indicated that having a street median in the middle of the road operates as a refuge and significantly reduced stress. Younger participants were (18-24 years) calmer than the relatively older participants (55-65 years). Arousal levels were higher when it came to the characteristics of the avatar (virtual pedestrian) in the simulation, especially for those avatars with adventurous traits. The pedestrian location influenced stress since the stress was higher on the street while crossing than waiting on the sidewalk. Significant causes of arousal were fear of accidents and an actual accident for pedestrians. The estimated random effects show a high degree of physical and mental learning by the participants while going through the scenarios.
LLM Pruning and Distillation in Practice: The Minitron Approach
Venue: arXiv cs.CL 2024
Year: 2024

We present a comprehensive report on compressing the Llama 3.1 8B and Mistral NeMo 12B models to 4B and 8B parameters, respectively, using pruning and distillation. We explore two distinct pruning strategies: (1) depth pruning and (2) joint hidden/attention/MLP (width) pruning, and evaluate the results on common benchmarks from the LM Evaluation Harness. The models are then aligned with NeMo Aligner and tested in instruct-tuned versions. This approach produces a compelling 4B model from Llama 3.1 8B and a state-of-the-art Mistral-NeMo-Minitron-8B (MN-Minitron-8B for brevity) model from Mistral NeMo 12B. We found that with no access to the original data, it is beneficial to slightly fine-tune teacher models on the distillation dataset. We open-source our base model weights on Hugging Face with a permissive license.
MIMA 2.0 -- Compact and Portable Multifunctional IoT integrated Menstrual Aid
Venue: arXiv cs.HC 2024
Year: 2024

The shredding intrauterine lining or the endometrium is known as Menstruation. It occurs every month and causes several issues like Menstrual Cramps and aches in the abdominal region, stains, menstrual malodor, rashes in intimate areas, and many more. In our research, almost all of the products available in the market do not cater to these problems single-handedly. There are few remedies available to cater to the cramps, among which heat therapy is the most commonly used. Our methodology, involved surveys regarding problems and the solutions to these problems that are deemed optimal. This inclusive approach helped us infer about the gaps in available menstrual aids which has become our guide towards developing MIMA (Multifunctional IoT Integrated Menstrual Aid). In this paper, we have featured an IOT incorporated multifunctional smart intimate wear that aims to provide for the multiple necessities of women during menstruation like leakproof, antibacterial, anti-odor, rash-free experience along with an integrated Bluetooth-controlled intimate heat-pad for relieving abdominal cramps. The entire process of product development has been done in phases according to feedback from target users in each stage. This paper is an extension to our paper [1] which serves as the proof of concept for our approach. The development has led us towards MIMA 2.0 featuring a completely concealed and integrated design that includes a safe Bluetooth-controlled heating system for the intimate area. The product has received incredibly positive feedback from survey participants.
Strategic AI adoption in SMEs: A Prescriptive Framework
Venue: arXiv cs.CY 2024
Year: 2024

Artificial Intelligence (AI) is increasingly acknowledged as a vital component for the advancement and competitiveness of modern organizations, including small and medium enterprises (SMEs). However, the adoption of AI technologies in SMEs faces significant barriers, primarily related to cost, lack of technical skills, and employee acceptance. This study proposes a comprehensive, phased framework designed to facilitate the effective adoption of AI in SMEs by systematically addressing these barriers. The framework begins with raising awareness and securing commitment from leadership, followed by the adoption of low-cost, general-purpose AI tools to build technical competence and foster a positive attitude towards AI. As familiarity with AI technologies increases, the framework advocates for the integration of task-specific AI tools to enhance efficiency and productivity. Subsequently, it guides organizations towards the in-house development of generative AI tools, providing greater customization and control. Finally, the framework addresses the development of discriminative AI models to meet highly specific and precision-oriented tasks. By providing a structured and incremental approach, this framework ensures that SMEs can navigate the complexities of AI integration effectively, driving innovation, efficiency, and competitive advantage. This study contributes to the field by offering a practical, prescriptive framework tailored to the unique needs of SMEs, facilitating the successful adoption of AI technologies and positioning these organizations for sustained growth in a competitive landscape.
SCREENER: A general framework for task-specific experiment design in quantitative MRI
Venue: arXiv cs.CV 2024
Year: 2024

Quantitative magnetic resonance imaging (qMRI) is increasingly investigated for use in a variety of clinical tasks from diagnosis, through staging, to treatment monitoring. However, experiment design in qMRI, the identification of the optimal acquisition protocols, has been focused on obtaining the most precise parameter estimations, with no regard for the specific requirements of downstream tasks. Here we propose SCREENER: A general framework for task-specific experiment design in quantitative MRI. SCREENER incorporates a task-specific objective and seeks the optimal protocol with a deep-reinforcement-learning (DRL) based optimization strategy. To illustrate this framework, we employ a task of classifying the inflammation status of bone marrow using diffusion MRI data with intravoxel incoherent motion (IVIM) modelling. Results demonstrate SCREENER outperforms previous ad hoc and optimized protocols under clinical signal-to-noise ratio (SNR) conditions, achieving significant improvement, both in binary classification tasks, e.g. from 67% to 89%, and in a multi-class classification task, from 46% to 59%. Additionally, we show this improvement is robust to the SNR. Lastly, we demonstrate the advantage of DRL-based optimization strategy, enabling zero-shot discovery of near-optimal protocols for a range of SNRs not used in training. In conclusion, SCREENER has the potential to enable wider uptake of qMRI in the clinic.
Joint PET-MRI Reconstruction with Diffusion Stochastic Differential Model
Venue: arXiv cs.CV 2024
Year: 2024

PET suffers from a low signal-to-noise ratio. Meanwhile, the k-space data acquisition process in MRI is time-consuming by PET-MRI systems. We aim to accelerate MRI and improve PET image quality. This paper proposed a novel joint reconstruction model by diffusion stochastic differential equations based on learning the joint probability distribution of PET and MRI. Compare the results underscore the qualitative and quantitative improvements our model brings to PET and MRI reconstruction, surpassing the current state-of-the-art methodologies. Joint PET-MRI reconstruction is a challenge in the PET-MRI system. This studies focused on the relationship extends beyond edges. In this study, PET is generated from MRI by learning joint probability distribution as the relationship.
LLaMA based Punctuation Restoration With Forward Pass Only Decoding
Venue: arXiv cs.CL 2024
Year: 2024

This paper introduces two advancements in the field of Large Language Model Annotation with a focus on punctuation restoration tasks. Our first contribution is the application of LLaMA for punctuation restoration, which demonstrates superior performance compared to the established benchmark. Despite its impressive quality, LLaMA faces challenges regarding inference speed and hallucinations. To address this, our second contribution presents Forward Pass Only Decoding (FPOD), a novel decoding approach for annotation tasks. This innovative method results in a substantial 19.8x improvement in inference speed, effectively addressing a critical bottleneck and enhancing the practical utility of LLaMA for large-scale data annotation tasks without hallucinations. The combination of these contributions not only solidifies LLaMA as a powerful tool for punctuation restoration but also highlights FPOD as a crucial strategy for overcoming speed constraints.
FactorLLM: Factorizing Knowledge via Mixture of Experts for Large Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Recent research has demonstrated that Feed-Forward Networks (FFNs) in Large Language Models (LLMs) play a pivotal role in storing diverse linguistic and factual knowledge. Conventional methods frequently face challenges due to knowledge confusion stemming from their monolithic and redundant architectures, which calls for more efficient solutions with minimal computational overhead, particularly for LLMs. In this paper, we explore the FFN computation paradigm in LLMs and introduce FactorLLM, a novel approach that decomposes well-trained dense FFNs into sparse sub-networks without requiring any further modifications, while maintaining the same level of performance. Furthermore, we embed a router from the Mixture-of-Experts (MoE), combined with our devised Prior-Approximate (PA) loss term that facilitates the dynamic activation of experts and knowledge adaptation, thereby accelerating computational processes and enhancing performance using minimal training data and fine-tuning steps. FactorLLM thus enables efficient knowledge factorization and activates select groups of experts specifically tailored to designated tasks, emulating the interactive functional segmentation of the human brain. Extensive experiments across various benchmarks demonstrate the effectiveness of our proposed FactorLLM which achieves comparable performance to the source model securing up to 85% model performance while obtaining over a 30% increase in inference speed. Code: https://github.com/zhenwuweihe/FactorLLM.
Improving embedding with contrastive fine-tuning on small datasets with expert-augmented scores
Venue: arXiv cs.CL 2024
Year: 2024

This paper presents an approach to improve text embedding models through contrastive fine-tuning on small datasets augmented with expert scores. It focuses on enhancing semantic textual similarity tasks and addressing text retrieval problems. The proposed method uses soft labels derived from expert-augmented scores to fine-tune embedding models, preserving their versatility and ensuring retrieval capability is improved. The paper evaluates the method using a Q\&A dataset from an online shopping website and eight expert models. Results show improved performance over a benchmark model across multiple metrics on various retrieval tasks from the massive text embedding benchmark (MTEB). The method is cost-effective and practical for real-world applications, especially when labeled data is scarce.
Ancient Wisdom, Modern Tools: Exploring Retrieval-Augmented LLMs for Ancient Indian Philosophy
Venue: arXiv cs.CL 2024
Year: 2024

LLMs have revolutionized the landscape of information retrieval and knowledge dissemination. However, their application in specialized areas is often hindered by factual inaccuracies and hallucinations, especially in long-tail knowledge distributions. We explore the potential of retrieval-augmented generation (RAG) models for long-form question answering (LFQA) in a specialized knowledge domain. We present VedantaNY-10M, a dataset curated from extensive public discourses on the ancient Indian philosophy of Advaita Vedanta. We develop and benchmark a RAG model against a standard, non-RAG LLM, focusing on transcription, retrieval, and generation performance. Human evaluations by computational linguists and domain experts show that the RAG model significantly outperforms the standard model in producing factual and comprehensive responses having fewer hallucinations. In addition, a keyword-based hybrid retriever that emphasizes unique low-frequency terms further improves results. Our study provides insights into effectively integrating modern large language models with ancient knowledge systems. Project page with dataset and code: https://sites.google.com/view/vedantany-10m
Higher-order Interpretations of Deepcode, a Learned Feedback Code
Venue: arXiv cs.IT 2024
Year: 2024

We present an interpretation of Deepcode, a learned feedback code that showcases higher-order error correction relative to an earlier interpretable model. By interpretation, we mean succinct analytical encoder and decoder expressions (albeit with learned parameters) in which the role of feedback in achieving error correction is easy to understand. By higher-order, we mean that longer sequences of large noise values are acted upon by the encoder (which has access to these through the feedback) and used in error correction at the decoder in a two-stage decoding process.
Explainable Anomaly Detection: Counterfactual driven What-If Analysis
Venue: arXiv cs.LG 2024
Year: 2024

There exists three main areas of study inside of the field of predictive maintenance: anomaly detection, fault diagnosis, and remaining useful life prediction. Notably, anomaly detection alerts the stakeholder that an anomaly is occurring. This raises two fundamental questions: what is causing the fault and how can we fix it? Inside of the field of explainable artificial intelligence, counterfactual explanations can give that information in the form of what changes to make to put the data point into the opposing class, in this case "healthy". The suggestions are not always actionable which may raise the interest in asking "what if we do this instead?" In this work, we provide a proof of concept for utilizing counterfactual explanations as what-if analysis. We perform this on the PRONOSTIA dataset with a temporal convolutional network as the anomaly detector. Our method presents the counterfactuals in the form of a what-if analysis for this base problem to inspire future work for more complex systems and scenarios.
A type system for data flow and alias analysis in ReScript
Venue: arXiv cs.PL 2024
Year: 2024

ReScript introduces a strongly typed language that targets JavaScript, as an alternative to gradually typed languages, such as TypeScript. In this paper, we present a type system for data-flow analysis for a subset of the ReScript language, more specific for a lambda-calculus with mutability and pattern matching. The type system is a local analysis that collects information about what variables are used and alias information.
Valuing an Engagement Surface using a Large Scale Dynamic Causal Model
Venue: arXiv cs.LG 2024
Year: 2024

With recent rapid growth in online shopping, AI-powered Engagement Surfaces (ES) have become ubiquitous across retail services. These engagement surfaces perform an increasing range of functions, including recommending new products for purchase, reminding customers of their orders and providing delivery notifications. Understanding the causal effect of engagement surfaces on value driven for customers and businesses remains an open scientific question. In this paper, we develop a dynamic causal model at scale to disentangle value attributable to an ES, and to assess its effectiveness. We demonstrate the application of this model to inform business decision-making by understanding returns on investment in the ES, and identifying product lines and features where the ES adds the most value.