OpenRaiser/Intern-Atlas huggingface dataset:
ExploreGen: Large Language Models for Envisioning the Uses and Risks of AI Technologies
Venue: arXiv cs.HC 2024
Year: 2024

Responsible AI design is increasingly seen as an imperative by both AI developers and AI compliance experts. One of the key tasks is envisioning AI technology uses and risks. Recent studies on the model and data cards reveal that AI practitioners struggle with this task due to its inherently challenging nature. Here, we demonstrate that leveraging a Large Language Model (LLM) can support AI practitioners in this task by enabling reflexivity, brainstorming, and deliberation, especially in the early design stages of the AI development process. We developed an LLM framework, ExploreGen, which generates realistic and varied uses of AI technology, including those overlooked by research, and classifies their risk level based on the EU AI Act regulation. We evaluated our framework using the case of Facial Recognition and Analysis technology in nine user studies with 25 AI practitioners. Our findings show that ExploreGen is helpful to both developers and compliance experts. They rated the uses as realistic and their risk classification as accurate (94.5%). Moreover, while unfamiliar with many of the uses, they rated them as having high adoption potential and transformational impact.
Enhancing Film Grain Coding in VVC: Improving Encoding Quality and Efficiency
Venue: arXiv cs.MM 2024
Year: 2024

This paper presents an in-depth analysis of film grain handling in open-source implementations of the Versatile Video Coding (VVC) standard. We focus on two key components: the Film Grain Analysis (FGA) module implemented in VVenC and the Film Grain Synthesis (FGS) module implemented in VVdeC. We describe the methodologies used to implement these modules and discuss the generation of Supplementary Enhancement Information (SEI) parameters to signal film grain characteristics in the encoded video sequences. Additionally, we conduct subjective and objective evaluations across Full HD videos to assess the effectiveness of film grain handling. Our results demonstrate the capability of the FGA and FGS techniques to accurately analyze and synthesize film grain, thereby improving the visual quality of encoded video content. Overall, our study contributes to advancing the understanding and implementation of film grain handling techniques in VVC open-source implementations, with implications for enhancing the viewing experience in multimedia applications.
Decoupled Edge Physics algorithms for collaborative XR simulations
Venue: arXiv cs.HC 2024
Year: 2024

This work proposes a novel approach to transform any modern game engine pipeline, for optimized performance and enhanced user experiences in Extended Reality (XR) environments. Decoupling the physics engine from the game engine pipeline and using a client-server N-1 architecture creates a scalable solution, efficiently serving multiple graphics clients on Head-Mounted Displays (HMDs) with a single physics engine on edge-cloud infrastructure. This approach ensures better synchronization in multiplayer scenarios without introducing overhead in single-player experiences, maintaining session continuity despite changes in user participation. Relocating the Physics Engine to an edge or cloud node reduces strain on local hardware, dedicating more resources to high-quality rendering and unlocking the full potential of untethered HMDs. We present four algorithms that decouple the physics engine, increasing frame rates and Quality of Experience (QoE) in VR simulations, supporting advanced interactions, numerous physics objects, and multi-user sessions with over 100 concurrent users. Incorporating a Geometric Algebra interpolator reduces inter-calls between dissected parts, maintaining QoE and easing network stress. Experimental validation, with more than 100 concurrent users, 10,000 physics objects, and softbody simulations, confirms the technical viability of the proposed architecture, showcasing transformative capabilities for more immersive and collaborative XR applications without compromising performance.
Easing Maintenance of Academic Static Analyzers
Venue: arXiv cs.PL 2024
Year: 2024

Academic research in static analysis produces software implementations. These implementations are time-consuming to develop and some need to be maintained in order to enable building further research upon the implementation. While necessary, these processes can be quickly challenging. This article documents the tools and techniques we have come up with to simplify the maintenance of Mopsa since 2017. Mopsa is a static analysis platform that aims at being sound. First, we describe an automated way to measure precision that does not require any baseline of true bugs obtained by manually inspecting the results. Further, it improves transparency of the analysis, and helps discovering regressions during continuous integration. Second, we have taken inspiration from standard tools observing the concrete execution of a program to design custom tools observing the abstract execution of the analyzed program itself, such as abstract debuggers and profilers. Finally, we report on some cases of automated testcase reduction.
Automate or Assist? The Role of Computational Models in Identifying Gendered Discourse in US Capital Trial Transcripts
Venue: arXiv cs.CL 2024
Year: 2024

The language used by US courtroom actors in criminal trials has long been studied for biases. However, systematic studies for bias in high-stakes court trials have been difficult, due to the nuanced nature of bias and the legal expertise required. Large language models offer the possibility to automate annotation. But validating the computational approach requires both an understanding of how automated methods fit in existing annotation workflows and what they really offer. We present a case study of adding a computational model to a complex and high-stakes problem: identifying gender-biased language in US capital trials for women defendants. Our team of experienced death-penalty lawyers and NLP technologists pursue a three-phase study: first annotating manually, then training and evaluating computational models, and finally comparing expert annotations to model predictions. Unlike many typical NLP tasks, annotating for gender bias in months-long capital trials is complicated, with many individual judgment calls. Contrary to standard arguments for automation that are based on efficiency and scalability, legal experts find the computational models most useful in providing opportunities to reflect on their own bias in annotation and to build consensus on annotation rules. This experience suggests that seeking to replace experts with computational models for complex annotation is both unrealistic and undesirable. Rather, computational models offer valuable opportunities to assist the legal experts in annotation-based studies.
On Initializing Transformers with Pre-trained Embeddings
Venue: arXiv cs.CL 2024
Year: 2024

It has become common practice now to use random initialization schemes, rather than the pre-trained embeddings, when training transformer based models from scratch. Indeed, we find that pre-trained word embeddings from GloVe, and some sub-word embeddings extracted from language models such as T5 and mT5 fare much worse compared to random initialization. This is counter-intuitive given the well-known representational and transfer-learning advantages of pre-training. Interestingly, we also find that BERT and mBERT embeddings fare better than random initialization, showing the advantages of pre-trained representations. In this work, we posit two potential factors that contribute to these mixed results: the model sensitivity to parameter distribution and the embedding interactions with position encodings. We observe that pre-trained GloVe, T5, and mT5 embeddings have a wider distribution of values. As argued in the initialization studies, such large value initializations can lead to poor training because of saturated outputs. Further, the larger embedding values can, in effect, absorb the smaller position encoding values when added together, thus losing position information. Standardizing the pre-trained embeddings to a narrow range (e.g. as prescribed by Xavier) leads to substantial gains for Glove, T5, and mT5 embeddings. On the other hand, BERT pre-trained embeddings, while larger, are still relatively closer to Xavier initialization range which may allow it to effectively transfer the pre-trained knowledge.
Struct-X: Enhancing Large Language Models Reasoning with Structured Data
Venue: arXiv cs.CL 2024
Year: 2024

Structured data, rich in logical and relational information, has the potential to enhance the reasoning abilities of large language models (LLMs). Still, its integration poses a challenge due to the risk of overwhelming LLMs with excessive tokens and irrelevant context information. To address this, we propose Struct-X, a novel framework that operates through five key phases: ``read-model-fill-reflect-reason'' efficiently enabling LLMs to utilize structured data. It begins by encoding structured data into a topological space using graph embeddings, followed by filling in missing entity information with knowledge retrieval modules, and filtering out irrelevant tokens via a self-supervised module. The final phase involves constructing a topological network with selected tokens to further reduce the total token length for more effective LLM inference. Additionally, Struct-X includes an Auxiliary Module trained to generate prompts, aiding LLMs in analyzing structured data. Extensive experiments on benchmarks, including the knowledge graph question-answer task and the long document reading comprehension task, show that Struct-X notably improves LLM reasoning, demonstrating the effectiveness of structured data augmentation in improving LLM inference with complex input context.
A Scheduler for Real-Time Service in Wi-Fi 8 Multi-AP Networks With Parameterized Spatial Reuse
Venue: arXiv cs.NI 2024
Year: 2024

Real-time applications (RTAs) require low delays and impose a significant challenge to Wi-Fi. In Wi-Fi, high delays are often caused by waiting for the channel to become idle. This problem can be solved with Parameterized Spatial Reuse (PSR), which allows a station (STA) to transmit its frame with reduced power simultaneously with a triggered uplink transmission in an overlapping network. The PSR opportunity depends on the pathloss between involved STAs, so the same transmission may allow PSR for one STA but not for another one. Thus, to satisfy tight delay constraints in dense overlapping networks, access points (APs) in the same area shall often allow PSR for every STA with RTA traffic. This letter proposes a fast scheduler enabling frequent PSR transmissions for RTA traffic. The scheduler uses Multi-AP coordination, the feature of upcoming Wi-Fi 8. With simulations, we show that it almost halves the delay for RTA traffic and does not deteriorate the quality of service for other traffic compared with an airtime fairness scheduler.
Computing: Looking Back and Moving Forward
Venue: arXiv cs.DC 2024
Year: 2024

The Internet and computer commercialization have transformed the computing systems area over the past sixty years, affecting society. Computer systems have evolved to meet diverse social needs thanks to technological advances. The Internet of Things (IoT), cloud computing, fog computing, edge computing, and other emerging paradigms provide new economic and creative potential. Therefore, this article explores and evaluates the elements impacting the advancement of computing platforms, including both long standing systems and frameworks and more recent innovations like cloud computing, quantum technology, and edge AI. In this article, we examine computing paradigms, domains, and next generation computing systems to better understand the past, present, and future of computing technologies. This paper provides readers with a comprehensive overview of developments in computing technologies and highlights promising research gaps for the advancement of future computing systems.
VegeDiff: Latent Diffusion Model for Geospatial Vegetation Forecasting
Venue: arXiv cs.CV 2024
Year: 2024

In the context of global climate change and frequent extreme weather events, forecasting future geospatial vegetation states under these conditions is of significant importance. The vegetation change process is influenced by the complex interplay between dynamic meteorological variables and static environmental variables, leading to high levels of uncertainty. Existing deterministic methods are inadequate in addressing this uncertainty and fail to accurately model the impact of these variables on vegetation, resulting in blurry and inaccurate forecasting results. To address these issues, we propose VegeDiff for the geospatial vegetation forecasting task. To our best knowledge, VegeDiff is the first to employ a diffusion model to probabilistically capture the uncertainties in vegetation change processes, enabling the generation of clear and accurate future vegetation states. VegeDiff also separately models the global impact of dynamic meteorological variables and the local effects of static environmental variables, thus accurately modeling the impact of these variables. Extensive experiments on geospatial vegetation forecasting tasks demonstrate the effectiveness of VegeDiff. By capturing the uncertainties in vegetation changes and modeling the complex influence of relevant variables, VegeDiff outperforms existing deterministic methods, providing clear and accurate forecasting results of future vegetation states. Interestingly, we demonstrate the potential of VegeDiff in applications of forecasting future vegetation states from multiple aspects and exploring the impact of meteorological variables on vegetation dynamics. The code of this work will be available at https://github.com/walking-shadow/ Official_VegeDiff.
Engineering Fully Dynamic Exact $Δ$-Orientation Algorithms
Venue: arXiv cs.DS 2024
Year: 2024

A (fully) dynamic graph algorithm is a data structure that supports edge insertions, edge deletions, and answers specific queries pertinent to the problem at hand. In this work, we address the fully dynamic edge orientation problem, also known as the fully dynamic $Δ$-orientation problem. The objective is to maintain an orientation of the edges in an undirected graph such that the out-degree of any vertex remains low. When edges are inserted or deleted, it may be necessary to reorient some edges to prevent vertices from having excessively high out-degrees. In this paper, we introduce the first algorithm that maintains an optimal edge orientation during both insertions and deletions. In experiments comparing with recent nearly exact algorithms, we achieve a 32% lower running time. The update time of our algorithm is up to 6 orders of magnitude faster than static exact algorithms.
Fusion Flow-enhanced Graph Pooling Residual Networks for Unmanned Aerial Vehicles Surveillance in Day and Night Dual Visions
Venue: arXiv cs.CV 2024
Year: 2024

Recognizing unauthorized Unmanned Aerial Vehicles (UAVs) within designated no-fly zones throughout the day and night is of paramount importance, where the unauthorized UAVs pose a substantial threat to both civil and military aviation safety. However, recognizing UAVs day and night with dual-vision cameras is nontrivial, since red-green-blue (RGB) images suffer from a low detection rate under an insufficient light condition, such as on cloudy or stormy days, while black-and-white infrared (IR) images struggle to capture UAVs that overlap with the background at night. In this paper, we propose a new optical flow-assisted graph-pooling residual network (OF-GPRN), which significantly enhances the UAV detection rate in day and night dual visions. The proposed OF-GPRN develops a new optical fusion to remove superfluous backgrounds, which improves RGB/IR imaging clarity. Furthermore, OF-GPRN extends optical fusion by incorporating a graph residual split attention network and a feature pyramid, which refines the perception of UAVs, leading to a higher success rate in UAV detection. A comprehensive performance evaluation is conducted using a benchmark UAV catch dataset. The results indicate that the proposed OF-GPRN elevates the UAV mean average precision (mAP) detection rate to 87.8%, marking a 17.9% advancement compared to the residual graph neural network (ResGCN)-based approach.
A SageMath Package for Elementary and Sign Vectors with Applications to Chemical Reaction Networks
Venue: arXiv cs.SC 2024
Year: 2024

We present our SageMath package elementary_vectors for computing elementary and sign vectors of real subspaces. In this setting, elementary vectors are support-minimal vectors that can be determined from maximal minors of a real matrix representing a subspace. By applying the sign function, we obtain the cocircuits of the corresponding oriented matroid, which in turn allow the computation of all sign vectors of a real subspace. As an application, we discuss sign vector conditions for existence and uniqueness of complex-balanced equilibria of chemical reaction networks with generalized mass-action kinetics. The conditions are formulated in terms of sign vectors of two subspaces arising from the stoichiometric coefficients and the kinetic orders of the reactions. We discuss how these conditions can be checked algorithmically, and we demonstrate the functionality of our package sign_vector_conditions in several examples.
Enhancing the Utility of Privacy-Preserving Cancer Classification using Synthetic Data
Venue: arXiv cs.CV 2024
Year: 2024

Deep learning holds immense promise for aiding radiologists in breast cancer detection. However, achieving optimal model performance is hampered by limitations in availability and sharing of data commonly associated to patient privacy concerns. Such concerns are further exacerbated, as traditional deep learning models can inadvertently leak sensitive training information. This work addresses these challenges exploring and quantifying the utility of privacy-preserving deep learning techniques, concretely, (i) differentially private stochastic gradient descent (DP-SGD) and (ii) fully synthetic training data generated by our proposed malignancy-conditioned generative adversarial network. We assess these methods via downstream malignancy classification of mammography masses using a transformer model. Our experimental results depict that synthetic data augmentation can improve privacy-utility tradeoffs in differentially private model training. Further, model pretraining on synthetic data achieves remarkable performance, which can be further increased with DP-SGD fine-tuning across all privacy guarantees. With this first in-depth exploration of privacy-preserving deep learning in breast imaging, we address current and emerging clinical privacy requirements and pave the way towards the adoption of private high-utility deep diagnostic models. Our reproducible codebase is publicly available at https://github.com/RichardObi/mammo_dp.
GraphMuse: A Library for Symbolic Music Graph Processing
Venue: arXiv cs.SD 2024
Year: 2024

Graph Neural Networks (GNNs) have recently gained traction in symbolic music tasks, yet a lack of a unified framework impedes progress. Addressing this gap, we present GraphMuse, a graph processing framework and library that facilitates efficient music graph processing and GNN training for symbolic music tasks. Central to our contribution is a new neighbor sampling technique specifically targeted toward meaningful behavior in musical scores. Additionally, GraphMuse integrates hierarchical modeling elements that augment the expressivity and capabilities of graph networks for musical tasks. Experiments with two specific musical prediction tasks -- pitch spelling and cadence detection -- demonstrate significant performance improvement over previous methods. Our hope is that GraphMuse will lead to a boost in, and standardization of, symbolic music processing based on graph representations. The library is available at https://github.com/manoskary/graphmuse
How to Mitigate the Dependencies of ChatGPT-4o in Engineering Education
Venue: arXiv cs.CY 2024
Year: 2024

The rapid evolution of large multimodal models (LMMs) has significantly impacted modern teaching and learning, especially in computer engineering. While LMMs offer extensive opportunities for enhancing learning, they also risk undermining traditional teaching methods and fostering excessive reliance on automated solutions. To counter this, we have developed strategies within curriculum to reduce the dependencies on LMMs that represented by ChatGPT-4o. These include designing course topics that encourage hands-on problem-solving. The proposed strategies were demonstrated through an actual course implementation. Preliminary results show that the methods effectively enhance student engagement and understanding, balancing the benefits of technology with the preservation of traditional learning principles.
Calibrated Diverse Ensemble Entropy Minimization for Robust Test-Time Adaptation in Prostate Cancer Detection
Venue: arXiv cs.CV 2024
Year: 2024

High resolution micro-ultrasound has demonstrated promise in real-time prostate cancer detection, with deep learning becoming a prominent tool for learning complex tissue properties reflected on ultrasound. However, a significant roadblock to real-world deployment remains, which prior works often overlook: model performance suffers when applied to data from different clinical centers due to variations in data distribution. This distribution shift significantly impacts the model's robustness, posing major challenge to clinical deployment. Domain adaptation and specifically its test-time adaption (TTA) variant offer a promising solution to address this challenge. In a setting designed to reflect real-world conditions, we compare existing methods to state-of-the-art TTA approaches adopted for cancer detection, demonstrating the lack of robustness to distribution shifts in the former. We then propose Diverse Ensemble Entropy Minimization (DEnEM), questioning the effectiveness of current TTA methods on ultrasound data. We show that these methods, although outperforming baselines, are suboptimal due to relying on neural networks output probabilities, which could be uncalibrated, or relying on data augmentation, which is not straightforward to define on ultrasound data. Our results show a significant improvement of $5\%$ to $7\%$ in AUROC over the existing methods and $3\%$ to $5\%$ over TTA methods, demonstrating the advantage of DEnEM in addressing distribution shift. \keywords{Ultrasound Imaging \and Prostate Cancer \and Computer-aided Diagnosis \and Distribution Shift Robustness \and Test-time Adaptation.}
Teleoperation in Robot-assisted MIS with Adaptive RCM via Admittance Control
Venue: arXiv cs.RO 2024
Year: 2024

This paper presents the development and assessment of a teleoperation framework for robot-assisted minimally invasive surgery (MIS). The framework leverages our novel integration of an adaptive remote center of motion (RCM) using admittance control. This framework operates within a redundancy resolution method specifically designed for the RCM constraint. We introduce a compact, low-cost, and modular custom-designed instrument module (IM) that ensures integration with the manipulator, featuring a force-torque sensor, a surgical instrument, and an actuation unit for driving the surgical instrument. The paper details the complete teleoperation framework, including the telemanipulation trajectory mapping, kinematic modelling, control strategy, and the integrated admittance controller. Finally, the system capability to perform various surgical tasks was demonstrated, including passing a thread through the rings, picking and placing objects, and trajectory tracking.