OpenRaiser/Intern-Atlas huggingface dataset:
A Performance Analysis Modeling Framework for Extended Reality Applications in Edge-Assisted Wireless Networks
Venue: arXiv cs.NI 2024
Year: 2024

Extended reality (XR) is at the center of attraction in the research community due to the emergence of augmented, mixed, and virtual reality applications. The performance of such applications needs to be uptight to maintain the requirements of latency, energy consumption, and freshness of data. Therefore, a comprehensive performance analysis model is required to assess the effectiveness of an XR application but is challenging to design due to the dependence of the performance metrics on several difficult-to-model parameters, such as computing resources and hardware utilization of XR and edge devices, which are controlled by both their operating systems and the application itself. Moreover, the heterogeneity in devices and wireless access networks brings additional challenges in modeling. In this paper, we propose a novel modeling framework for performance analysis of XR applications considering edge-assisted wireless networks and validate the model with experimental data collected from testbeds designed specifically for XR applications. In addition, we present the challenges associated with performance analysis modeling and present methods to overcome them in detail. Finally, the performance evaluation shows that the proposed analytical model can analyze XR applications' performance with high accuracy compared to the state-of-the-art analytical models.
Towards an Accessible and Rapidly Trainable Rhythm Sequencer Using a Generative Stacked Autoencoder
Venue: arXiv cs.SD 2024
Year: 2024

Neural networks and deep learning are often deployed for the sake of the most comprehensive music generation with as little involvement as possible from the human musician. Implementations in aid of, or being a tool for, music practitioners are sparse. This paper proposes the integration of generative stacked autoencoder structures for rhythm generation, within a conventional melodic step-sequencer. It further aims to work towards its implementation being accessible to the average electronic music practitioner. Several model architectures have been trained and tested for their creative potential. While the currently implementations do display limitations, they do represent viable creative solutions for music practitioners.
LUCID: A Framework for Reducing False Positives and Inconsistencies Among Container Scanning Tools
Venue: arXiv cs.CR 2024
Year: 2024

Containerization has emerged as a revolutionary technology in the software development and deployment industry. Containers offer a portable and lightweight solution that allows for packaging applications and their dependencies systematically and efficiently. In addition, containers offer faster deployment and near-native performance with isolation and security drawbacks compared to Virtual Machines. To address the security issues, scanning tools that scan containers for preexisting vulnerabilities have been developed, but they suffer from false positives. Moreover, using different scanning tools to scan the same container provides different results, which leads to inconsistencies and confusion. Limited work has been done to address these issues. This paper provides a fully functional and extensible framework named LUCID that can reduce false positives and inconsistencies provided by multiple scanning tools. We use a database-centric approach and perform query-based analysis, to pinpoint the causes for inconsistencies. Our results show that our framework can reduce inconsistencies by 70%. The framework has been tested on both Intel64/AMD64 and ARM architecture. We also create a Dynamic Classification component that can successfully classify and predict the different severity levels with an accuracy of 84%. We believe this paper will raise awareness regarding security in container technologies and enable container scanning companies to improve their tool to provide better and more consistent results.
Auditing an Automatic Grading Model with deep Reinforcement Learning
Venue: arXiv cs.AI 2024
Year: 2024

We explore the use of deep reinforcement learning to audit an automatic short answer grading (ASAG) model. Automatic grading may decrease the time burden of rating open-ended items for educators, but a lack of robust evaluation methods for these models can result in uncertainty of their quality. Current state-of-the-art ASAG models are configured to match human ratings from a training set, and researchers typically assess their quality with accuracy metrics that signify agreement between model and human scores. In this paper, we show that a high level of agreement to human ratings does not give sufficient evidence that an ASAG model is infallible. We train a reinforcement learning agent to revise student responses with the objective of achieving a high rating from an automatic grading model in the least number of revisions. By analyzing the agent's revised responses that achieve a high grade from the ASAG model but would not be considered a high scoring responses according to a scoring rubric, we discover ways in which the automated grader can be exploited, exposing shortcomings in the grading model.
MUD: Towards a Large-Scale and Noise-Filtered UI Dataset for Modern Style UI Modeling
Venue: arXiv cs.HC 2024
Year: 2024

The importance of computational modeling of mobile user interfaces (UIs) is undeniable. However, these require a high-quality UI dataset. Existing datasets are often outdated, collected years ago, and are frequently noisy with mismatches in their visual representation. This presents challenges in modeling UI understanding in the wild. This paper introduces a novel approach to automatically mine UI data from Android apps, leveraging Large Language Models (LLMs) to mimic human-like exploration. To ensure dataset quality, we employ the best practices in UI noise filtering and incorporate human annotation as a final validation step. Our results demonstrate the effectiveness of LLMs-enhanced app exploration in mining more meaningful UIs, resulting in a large dataset MUD of 18k human-annotated UIs from 3.3k apps. We highlight the usefulness of MUD in two common UI modeling tasks: element detection and UI retrieval, showcasing its potential to establish a foundation for future research into high-quality, modern UIs.
Vertex Shader Domain Warping with Automatic Differentiation
Venue: arXiv cs.GR 2024
Year: 2024

Domain warping is a technique commonly used in creative coding to distort graphics and add visual interest to a work. The approach has the potential to be used in 3D art as mesh vertices can be efficiently warped using a vertex shader in a WebGL pipeline. However, 3D models packaged for the web typically come with baked-in normal vectors, and these need to be updated when vertex positions change for lighting calculations to work. This is typically done via finite differences, which requires parameter tuning to achieve optimal visual fidelity. We present a method for 3D domain warping that works with automatic differentiation, allowing exact normals to be used without any tuning while still benefiting from hardware acceleration.
Post Training Quantization of Large Language Models with Microscaling Formats
Venue: arXiv cs.LG 2024
Year: 2024

Large Language Models (LLMs) have distinguished themselves with outstanding performance in complex language modeling tasks, yet they come with significant computational and storage challenges. This paper explores the potential of quantization to mitigate these challenges. We systematically study the combined application of three well-known post-training techniques, SmoothQuant, AWQ, and GPTQ, and provide a comprehensive analysis of their interactions and implications for advancing LLM quantization. We enhance the versatility of these methods by enabling quantization to microscaling (MX) formats, extending the applicability of these PTQ algorithms beyond their original fixed-point format targets. We show that combining different PTQ methods enables us to quantize models to 4-bit weights and 8-bit activations using the MXINT format with negligible accuracy loss compared to the uncompressed baseline.
Enhanced Online Test-time Adaptation with Feature-Weight Cosine Alignment
Venue: arXiv cs.CV 2024
Year: 2024

Online Test-Time Adaptation (OTTA) has emerged as an effective strategy to handle distributional shifts, allowing on-the-fly adaptation of pre-trained models to new target domains during inference, without the need for source data. We uncovered that the widely studied entropy minimization (EM) method for OTTA, suffers from noisy gradients due to ambiguity near decision boundaries and incorrect low-entropy predictions. To overcome these limitations, this paper introduces a novel cosine alignment optimization approach with a dual-objective loss function that refines the precision of class predictions and adaptability to novel domains. Specifically, our method optimizes the cosine similarity between feature vectors and class weight vectors, enhancing the precision of class predictions and the model's adaptability to novel domains. Our method outperforms state-of-the-art techniques and sets a new benchmark in multiple datasets, including CIFAR-10-C, CIFAR-100-C, ImageNet-C, Office-Home, and DomainNet datasets, demonstrating high accuracy and robustness against diverse corruptions and domain shifts.
SPEAK: Speech-Driven Pose and Emotion-Adjustable Talking Head Generation
Venue: arXiv cs.CV 2024
Year: 2024

Most earlier researches on talking face generation have focused on the synchronization of lip motion and speech content. However, head pose and facial emotions are equally important characteristics of natural faces. While audio-driven talking face generation has seen notable advancements, existing methods either overlook facial emotions or are limited to specific individuals and cannot be applied to arbitrary subjects. In this paper, we propose a novel one-shot Talking Head Generation framework (SPEAK) that distinguishes itself from the general Talking Face Generation by enabling emotional and postural control. Specifically, we introduce Inter-Reconstructed Feature Disentanglement (IRFD) module to decouple facial features into three latent spaces. Then we design a face editing module that modifies speech content and facial latent codes into a single latent space. Subsequently, we present a novel generator that employs modified latent codes derived from the editing module to regulate emotional expression, head poses, and speech content in synthesizing facial animations. Extensive trials demonstrate that our method ensures lip synchronization with the audio while enabling decoupled control of facial features, it can generate realistic talking head with coordinated lip motions, authentic facial emotions, and smooth head movements. The demo video is available: https://anonymous.4open.science/r/SPEAK-8A22
Robust Beamforming with Gradient-based Liquid Neural Network
Venue: arXiv cs.IT 2024
Year: 2024

Millimeter-wave (mmWave) multiple-input multiple-output (MIMO) communication with the advanced beamforming technologies is a key enabler to meet the growing demands of future mobile communication. However, the dynamic nature of cellular channels in large-scale urban mmWave MIMO communication scenarios brings substantial challenges, particularly in terms of complexity and robustness. To address these issues, we propose a robust gradient-based liquid neural network (GLNN) framework that utilizes ordinary differential equation-based liquid neurons to solve the beamforming problem. Specifically, our proposed GLNN framework takes gradients of the optimization objective function as inputs to extract the high-order channel feature information, and then introduces a residual connection to mitigate the training burden. Furthermore, we use the manifold learning technique to compress the search space of the beamforming problem. These designs enable the GLNN to effectively maintain low complexity while ensuring strong robustness to noisy and highly dynamic channels. Extensive simulation results demonstrate that the GLNN can achieve 4.15% higher spectral efficiency than that of typical iterative algorithms, and reduce the time consumption to only 1.61% that of conventional methods.
Finding a Way Through the Social Media Labyrinth: Guiding Design Through User Expectations
Venue: arXiv cs.HC 2024
Year: 2024

Social networking services (SNS) have become integral to modern life to create and maintain meaningful relationships. Nevertheless, their historic growth of features has led to labyrinthine user interfaces (UIs) that often result in frustration among users - for instance, when trying to control privacy-related settings. This paper aims to mitigate labyrinthine UIs by studying users' expectations (N=21) through an online card sorting exercise based on 58 common SNS UI features, teaching us about their expectations regarding the importance of specific UI features and the frequency with which they use them. Our findings offer a valuable understanding of the relationship between the importance and frequency of UI features and provide design considerations for six identified UI feature groups. Through these findings, we inform the design and development of user-centred alternatives to current SNS interfaces that enable users to successfully navigate SNS and feel in control over their data by meeting their expectations.
A Standard Rigid Transformation Notation Convention for Robotics Research
Venue: arXiv cs.RO 2024
Year: 2024

Notation conventions for rigid transformations are as diverse as they are fundamental to the field of robotics. A well-defined convention that is practical, consistent and unambiguous is essential for the clear communication of ideas and to foster collaboration between researchers. This work presents an analysis of conventions used in state-of-the-art robotics research, defines a new notation convention, and provides software packages to facilitate its use. To shed some light on the current state of notation conventions in robotics research, this work presents an analysis of the ICRA 2023 proceedings, focusing on the notation conventions used for rigid transformations. A total of 1655 papers were inspected to identify the convention used, and key insights about trends and usage preferences are derived. Based on this analysis, a new notation convention called RIGID is defined, which complies with the "ISO 80000 Standard on Quantities and Units". The RIGID convention is designed to be concise yet unambiguous and easy to use. Additionally, this work introduces a LaTeX package that facilitates the use of the RIGID notation in manuscripts preparation through simple customizable commands that can be easily translated into variable names for software development.
Forecasting with an N-dimensional Langevin Equation and a Neural-Ordinary Differential Equation
Venue: arXiv cs.LG 2024
Year: 2024

Accurate prediction of electricity day-ahead prices is essential in competitive electricity markets. Although stationary electricity-price forecasting techniques have received considerable attention, research on non-stationary methods is comparatively scarce, despite the common prevalence of non-stationary features in electricity markets. Specifically, existing non-stationary techniques will often aim to address individual non-stationary features in isolation, leaving aside the exploration of concurrent multiple non-stationary effects. Our overarching objective here is the formulation of a framework to systematically model and forecast non-stationary electricity-price time series, encompassing the broader scope of non-stationary behavior. For this purpose we develop a data-driven model that combines an N-dimensional Langevin equation (LE) with a neural-ordinary differential equation (NODE). The LE captures fine-grained details of the electricity-price behavior in stationary regimes but is inadequate for non-stationary conditions. To overcome this inherent limitation, we adopt a NODE approach to learn, and at the same time predict, the difference between the actual electricity-price time series and the simulated price trajectories generated by the LE. By learning this difference, the NODE reconstructs the non-stationary components of the time series that the LE is not able to capture. We exemplify the effectiveness of our framework using the Spanish electricity day-ahead market as a prototypical case study. Our findings reveal that the NODE nicely complements the LE, providing a comprehensive strategy to tackle both stationary and non-stationary electricity-price behavior. The framework's dependability and robustness is demonstrated through different non-stationary scenarios by comparing it against a range of basic naive methods.
A New Algorithm for Computing $α$-Capacity
Venue: arXiv cs.IT 2024
Year: 2024

The problem of computing $α$-capacity for $α>1$ is equivalent to that of computing the correct decoding exponent. Various algorithms for computing them have been proposed, such as Arimoto and Jitsumatsu--Oohama algorithm. In this study, we propose a novel alternating optimization algorithm for computing the $α$-capacity for $α>1$ based on a variational characterization of the Augustin--Csisz{á}r mutual information. A comparison of the convergence performance of these algorithms is demonstrated through numerical examples.
Hell is Paved with Good Intentions: The Intricate Relationship Between Cognitive Biases and Dark Patterns
Venue: arXiv cs.HC 2024
Year: 2024

Throughout the past decade, research in HCI has identified numerous instances of dark patterns in digital interfaces. These efforts have led to a well-fostered typology describing harmful strategies users struggle to navigate. However, an in-depth understanding of the underlying mechanisms that deceive, coerce, or manipulate users is missing. We explore the interplay between cognitive biases and dark patterns to address this gap. To that end, we conducted four focus groups with experts (N=15) in psychology and dark pattern scholarship, inquiring how they conceptualise the relation between cognitive biases and dark patterns. Based on our results, we constructed the "Relationship Model of Cognitive Biases and Dark Patterns" which illustrates how cognitive bias and deceptive design patterns relate and identifies opportune moments for ethical reconsideration and user protection mechanisms. Our insights contribute to the current discourse by emphasising ethical design decisions and their implications in the field of HCI.
Networked Control with Hybrid Automatic Repeat Request Protocols
Venue: arXiv cs.IT 2024
Year: 2024

We study feedback control of a dynamical process over a lossy channel equipped with a hybrid automatic repeat request protocol that connects a sensor to an actuator. The dynamical process is modeled by a Gauss-Markov process, and the lossy channel by a packet-erasure channel with ideal feedback. We suppose that data is communicated in the format of packets with negligible quantization error. In such a networked control system, whenever a packet loss occurs, there exists a tradeoff between transmitting new sensory information with a lower success probability and retransmitting previously failed sensory information with a higher success probability. In essence, an inherent tradeoff between freshness and reliability. To address this tradeoff, we consider a linear-quadratic-regulator performance index, which penalizes state deviations and control efforts over a finite horizon, and jointly design optimal policies for an encoder and a decoder, which are collocated with the sensor and the actuator, respectively. Our emphasis here lies specifically on designing switching and control policies, rather than error-correcting codes. We derive the structural properties of the optimal encoding and decoding policies. We show that the former is a threshold switching policy and the latter is a certainty-equivalent control policy. In addition, we specify the iterative equations that the encoder and the decoder need to solve in order to implement the optimal policies.
AnyRotate: Gravity-Invariant In-Hand Object Rotation with Sim-to-Real Touch
Venue: arXiv cs.RO 2024
Year: 2024

Human hands are capable of in-hand manipulation in the presence of different hand motions. For a robot hand, harnessing rich tactile information to achieve this level of dexterity still remains a significant challenge. In this paper, we present AnyRotate, a system for gravity-invariant multi-axis in-hand object rotation using dense featured sim-to-real touch. We tackle this problem by training a dense tactile policy in simulation and present a sim-to-real method for rich tactile sensing to achieve zero-shot policy transfer. Our formulation allows the training of a unified policy to rotate unseen objects about arbitrary rotation axes in any hand direction. In our experiments, we highlight the benefit of capturing detailed contact information when handling objects of varying properties. Interestingly, we found rich multi-fingered tactile sensing can detect unstable grasps and provide a reactive behavior that improves the robustness of the policy. The project website can be found at https://maxyang27896.github.io/anyrotate/.
Semi-Supervised Weed Detection for Rapid Deployment and Enhanced Efficiency
Venue: arXiv cs.CV 2024
Year: 2024

Weeds present a significant challenge in agriculture, causing yield loss and requiring expensive control measures. Automatic weed detection using computer vision and deep learning offers a promising solution. However, conventional deep learning methods often require large amounts of labelled training data, which can be costly and time-consuming to acquire. This paper introduces a novel method for semi-supervised weed detection, comprising two main components. Firstly, a multi-scale feature representation technique is employed to capture distinctive weed features across different scales. Secondly, we propose an adaptive pseudo-label assignment strategy, leveraging a small set of labelled images during training. This strategy dynamically assigns confidence scores to pseudo-labels generated from unlabeled data. Additionally, our approach integrates epoch-corresponding and mixed pseudo-labels to further enhance the learning process. Experimental results on the COCO dataset and five prominent weed datasets -- CottonWeedDet12, CropAndWeed, Palmer amaranth, RadishWheat, and RoboWeedMap -- illustrate that our method achieves state-of-the-art performance in weed detection, even with significantly less labelled data compared to existing techniques. This approach holds the potential to alleviate the labelling burden and enhance the feasibility and deployment speed of deep learning for weed detection in real-world agricultural scenarios.