OpenRaiser/Intern-Atlas huggingface dataset:
The Concatenator: A Bayesian Approach To Real Time Concatenative Musaicing
Venue: arXiv cs.SD 2024
Year: 2024

We present ``The Concatenator,'' a real time system for audio-guided concatenative synthesis. Similarly to Driedger et al.'s ``musaicing'' (or ``audio mosaicing'') technique, we concatenate a set number of windows within a corpus of audio to re-create the harmonic and percussive aspects of a target audio stream. Unlike Driedger's NMF-based technique, however, we instead use an explicitly Bayesian point of view, where corpus window indices are hidden states and the target audio stream is an observation. We use a particle filter to infer the best hidden corpus states in real-time. Our transition model includes a tunable parameter to control the time-continuity of corpus grains, and our observation model allows users to prioritize how quickly windows change to match the target. Because the computational complexity of the system is independent of the corpus size, our system scales to corpora that are hours long, which is an important feature in the age of vast audio data collections. Within The Concatenator module itself, composers can vary grain length, fit to target, and pitch shift in real time while reacting to the sounds they hear, enabling them to rapidly iterate ideas. To conclude our work, we evaluate our system with extensive quantitative tests of the effects of parameters, as well as a qualitative evaluation with artistic insights. Based on the quality of the results, we believe the real-time capability unlocks new avenues for musical expression and control, suitable for live performance and modular synthesis integration, which furthermore represents an essential breakthrough in concatenative synthesis technology.
Improved Maximin Share Approximations for Chores by Bin Packing
Venue: arXiv cs.GT 2024
Year: 2024

We study fair division of indivisible chores among $n$ agents with additive cost functions using the popular fairness notion of maximin share (MMS). Since MMS allocations do not always exist for more than two agents, the goal has been to improve its approximations and identify interesting special cases where MMS allocations exists. We show the existence of 1) 1-out-of-$\lfloor \frac{9}{11}n\rfloor$ MMS allocations, which improves the state-of-the-art factor of 1-out-of-$\lfloor \frac{3}{4}n\rfloor$. 2) MMS allocations for factored instances, which resolves an open question posed by Ebadian et al. (2021). 3) $15/13$-MMS allocations for personalized bivalued instances, improving the state-of-the-art factor of $13/11$. We achieve these results by leveraging the HFFD algorithm of Huang and Lu (2021). Our approach also provides polynomial-time algorithms for computing an MMS allocation for factored instances and a $15/13$-MMS allocation for personalized bivalued instances.
The Survey of Chiplet-based Integrated Architecture: An EDA perspective
Venue: arXiv cs.AR 2024
Year: 2024

Enhancing performance while reducing costs is the fundamental design philosophy of integrated circuits (ICs). With advancements in packaging technology, interposer-based chiplet architecture has emerged as a promising solution. Chiplet integration, often referred to as 2.5D IC, offers significant benefits, including cost-effectiveness, reusability, and improved performance. However, realizing these advantages heavily relies on effective electronic design automation (EDA) processes. EDA plays a crucial role in optimizing architecture design, partitioning, combination, physical design, reliability analysis, etc. Currently, optimizing the automation methodologies for chiplet architecture is a popular focus; therefore, we propose a survey to summarize current methods and discuss future directions. This paper will review the research literature on design automation methods for chiplet-based architectures, highlighting current challenges and exploring opportunities in 2.5D IC from an EDA perspective. We expect this survey will provide valuable insights for the future development of EDA tools for chiplet-based integrated architectures.
Bayesian Calibration of Win Rate Estimation with LLM Evaluators
Venue: arXiv cs.CL 2024
Year: 2024

Recent advances in large language models (LLMs) show the potential of using LLMs as evaluators for assessing the quality of text generations from LLMs. However, applying LLM evaluators naively to compare or judge between different systems can lead to unreliable results due to the intrinsic win rate estimation bias of LLM evaluators. In order to mitigate this problem, we propose two calibration methods, Bayesian Win Rate Sampling (BWRS) and Bayesian Dawid-Skene, both of which leverage Bayesian inference to more accurately infer the true win rate of generative language models. We empirically validate our methods on six datasets covering story generation, summarization, and instruction following tasks. We show that both our methods are effective in improving the accuracy of win rate estimation using LLMs as evaluators, offering a promising direction for reliable automatic text quality evaluation.
ML-Promise: A Multilingual Dataset for Corporate Promise Verification
Venue: arXiv cs.CL 2024
Year: 2024

Promises made by politicians, corporate leaders, and public figures have a significant impact on public perception, trust, and institutional reputation. However, the complexity and volume of such commitments, coupled with difficulties in verifying their fulfillment, necessitate innovative methods for assessing their credibility. This paper introduces the concept of Promise Verification, a systematic approach involving steps such as promise identification, evidence assessment, and the evaluation of timing for verification. We propose the first multilingual dataset, ML-Promise, which includes English, French, Chinese, Japanese, and Korean, aimed at facilitating in-depth verification of promises, particularly in the context of Environmental, Social, and Governance (ESG) reports. Given the growing emphasis on corporate environmental contributions, this dataset addresses the challenge of evaluating corporate promises, especially in light of practices like greenwashing. Our findings also explore textual and image-based baselines, with promising results from retrieval-augmented generation (RAG) approaches. This work aims to foster further discourse on the accountability of public commitments across multiple languages and domains.
Smoke Screens and Scapegoats: The Reality of General Data Protection Regulation Compliance -- Privacy and Ethics in the Case of Replika AI
Venue: arXiv cs.CY 2024
Year: 2024

Currently artificial intelligence (AI)-enabled chatbots are capturing the hearts and imaginations of the public at large. Chatbots that users can build and personalize, as well as pre-designed avatars ready for users' selection, all of these are on offer in applications to provide social companionship, friends and even love. These systems, however, have demonstrated challenges on the privacy and ethics front. This paper takes a critical approach towards examining the intricacies of these issues within AI companion services. We chose Replika as a case and employed close reading to examine the service's privacy policy. We additionally analyze articles from public media about the company and its practices to gain insight into the trustworthiness and integrity of the information provided in the policy. The aim is to ascertain whether seeming General Data Protection Regulation (GDPR) compliance equals reliability of required information, or whether the area of GDPR compliance in itself is one riddled with ethical challenges. The paper contributes to a growing body of scholarship on ethics and privacy related matters in the sphere of social chatbots. The results reveal that despite privacy notices, data collection practices might harvest personal data without users' full awareness. Cross-textual comparison reveals that privacy notice information does not fully correspond with other information sources.
Memory Remedy: An AI-Enhanced Interactive Story Exploring Human-Robot Interaction and Companionship
Venue: arXiv cs.HC 2024
Year: 2024

We present our approach to using AI-generated content (AIGC) and multiple media to develop an immersive, game-based, interactive story experience. The narrative of the story, "Memory Remedy", unfolds through flashbacks, allowing the audience to gradually uncover the story and the complex relationship between the robot protagonist and the older adults. This exploration explores important themes such as the journey of life, the profound influence of memories, and the concept of post-human emotional care. By engaging with this AIGC-based interactive story, audiences are encouraged to reflect on the potential role of robotic companionship in the lives of older adults in the future; and to encourage deeper reflection on the complex relationship between artificial intelligence and humanity.
Pose2Trajectory: Using Transformers on Body Pose to Predict Tennis Player's Trajectory
Venue: arXiv cs.CV 2024
Year: 2024

Tracking the trajectory of tennis players can help camera operators in production. Predicting future movement enables cameras to automatically track and predict a player's future trajectory without human intervention. Predicting future human movement in the context of complex physical tasks is also intellectually satisfying. Swift advancements in sports analytics and the wide availability of videos for tennis have inspired us to propose a novel method called Pose2Trajectory, which predicts a tennis player's future trajectory as a sequence derived from their body joints' data and ball position. Demonstrating impressive accuracy, our approach capitalizes on body joint information to provide a comprehensive understanding of the human body's geometry and motion, thereby enhancing the prediction of the player's trajectory. We use encoder-decoder Transformer architecture trained on the joints and trajectory information of the players with ball positions. The predicted sequence can provide information to help close-up cameras to keep tracking the tennis player, following centroid coordinates. We generate a high-quality dataset from multiple videos to assist tennis player movement prediction using object detection and human pose estimation methods. It contains bounding boxes and joint information for tennis players and ball positions in singles tennis games. Our method shows promising results in predicting the tennis player's movement trajectory with different sequence prediction lengths using the joints and trajectory information with the ball position.
Joint wireless and computing resource management with optimal slice selection in in-network-edge metaverse system
Venue: arXiv cs.DC 2024
Year: 2024

This paper presents an approach to joint wireless and computing resource management in slice-enabled metaverse networks, addressing the challenges of inter-slice and intra-slice resource allocation in the presence of in-network computing. We formulate the problem as a mixed-integer nonlinear programming (MINLP) problem and derive an optimal solution using standard optimization techniques. Through extensive simulations, we demonstrate that our proposed method significantly improves system performance by effectively balancing the allocation of radio and computing resources across multiple slices. Our approach outperforms existing benchmarks, particularly in scenarios with high user demand and varying computational tasks.
Impact of Label Noise on Learning Complex Features
Venue: arXiv cs.LG 2024
Year: 2024

Neural networks trained with stochastic gradient descent exhibit an inductive bias towards simpler decision boundaries, typically converging to a narrow family of functions, and often fail to capture more complex features. This phenomenon raises concerns about the capacity of deep models to adequately learn and represent real-world datasets. Traditional approaches such as explicit regularization, data augmentation, architectural modifications, etc., have largely proven ineffective in encouraging the models to learn diverse features. In this work, we investigate the impact of pre-training models with noisy labels on the dynamics of SGD across various architectures and datasets. We show that pretraining promotes learning complex functions and diverse features in the presence of noise. Our experiments demonstrate that pre-training with noisy labels encourages gradient descent to find alternate minima that do not solely depend upon simple features, rather learns more complex and broader set of features, without hurting performance.
Multistage Fine-tuning Strategies for Automatic Speech Recognition in Low-resource Languages
Venue: arXiv cs.CL 2024
Year: 2024

This paper presents a novel multistage fine-tuning strategy designed to enhance automatic speech recognition (ASR) performance in low-resource languages using OpenAI's Whisper model. In this approach we aim to build ASR model for languages with limited digital resources by sequentially adapting the model across linguistically similar languages. We experimented this on the Malasar language, a Dravidian language spoken by approximately ten thousand people in the Western Ghats of South India. Malasar language faces critical challenges for technological intervention due to its lack of a native script and absence of digital or spoken data resources. Working in collaboration with Wycliffe India and Malasar community members, we created a spoken Malasar corpus paired with transcription in Tamil script, a closely related major language. In our approach to build ASR model for Malasar, we first build an intermediate Tamil ASR, leveraging higher data availability for Tamil annotated speech. This intermediate model is subsequently fine-tuned on Malasar data, allowing for more effective ASR adaptation despite limited resources. The multistage fine-tuning strategy demonstrated significant improvements over direct fine-tuning on Malasar data alone, achieving a word error rate (WER) of 51.9%, which is 4.5% absolute reduction when compared to the direct fine-tuning method. Further a WER reduction to 47.3% was achieved through punctuation removal in post-processing, which addresses formatting inconsistencies that impact evaluation. Our results underscore the effectiveness of sequential multistage fine-tuning combined with targeted post-processing as a scalable strategy for ASR system development in low-resource languages, especially where linguistic similarities can be leveraged to bridge gaps in training data.
Brain Tumour Removing and Missing Modality Generation using 3D WDM
Venue: arXiv cs.CV 2024
Year: 2024

This paper presents the second-placed solution for task 8 and the participation solution for task 7 of BraTS 2024. The adoption of automated brain analysis algorithms to support clinical practice is increasing. However, many of these algorithms struggle with the presence of brain lesions or the absence of certain MRI modalities. The alterations in the brain's morphology leads to high variability and thus poor performance of predictive models that were trained only on healthy brains. The lack of information that is usually provided by some of the missing MRI modalities also reduces the reliability of the prediction models trained with all modalities. In order to improve the performance of these models, we propose the use of conditional 3D wavelet diffusion models. The wavelet transform enabled full-resolution image training and prediction on a GPU with 48 GB VRAM, without patching or downsampling, preserving all information for prediction. The code for these tasks is available at https://github.com/ShadowTwin41/BraTS_2023_2024_solutions.
Cybercrime Prediction via Geographically Weighted Learning
Venue: arXiv cs.LG 2024
Year: 2024

Inspired by the success of Geographically Weighted Regression and its accounting for spatial variations, we propose GeogGNN -- A graph neural network model that accounts for geographical latitude and longitudinal points. Using a synthetically generated dataset, we apply the algorithm for a 4-class classification problem in cybersecurity with seemingly realistic geographic coordinates centered in the Gulf Cooperation Council region. We demonstrate that it has higher accuracy than standard neural networks and convolutional neural networks that treat the coordinates as features. Encouraged by the speed-up in model accuracy by the GeogGNN model, we provide a general mathematical result that demonstrates that a geometrically weighted neural network will, in principle, always display higher accuracy in the classification of spatially dependent data by making use of spatial continuity and local averaging features.
Explainable Search and Discovery of Visual Cultural Heritage Collections with Multimodal Large Language Models
Venue: arXiv cs.CV 2024
Year: 2024

Many cultural institutions have made large digitized visual collections available online, often under permissible re-use licences. Creating interfaces for exploring and searching these collections is difficult, particularly in the absence of granular metadata. In this paper, we introduce a method for using state-of-the-art multimodal large language models (LLMs) to enable an open-ended, explainable search and discovery interface for visual collections. We show how our approach can create novel clustering and recommendation systems that avoid common pitfalls of methods based directly on visual embeddings. Of particular interest is the ability to offer concrete textual explanations of each recommendation without the need to preselect the features of interest. Together, these features can create a digital interface that is more open-ended and flexible while also being better suited to addressing privacy and ethical concerns. Through a case study using a collection of documentary photographs, we provide several metrics showing the efficacy and possibilities of our approach.
Socially-Aware Opinion-Based Navigation with Oval Limit Cycles
Venue: arXiv cs.RO 2024
Year: 2024

When humans move in a shared space, they choose navigation strategies that preserve their mutual safety. At the same time, each human seeks to minimise the number of modifications to her/his path. In order to achieve this result, humans use unwritten rules and reach a consensus on their decisions about the motion direction by exchanging non-verbal messages. They then implement their choice in a mutually acceptable way. Socially-aware navigation denotes a research effort aimed at replicating this logic inside robots. Existing results focus either on how robots can participate in negotiations with humans, or on how they can move in a socially acceptable way. We propose a holistic approach in which the two aspects are jointly considered. Specifically, we show that by combining opinion dynamics (to reach a consensus) with vortex fields (to generate socially acceptable trajectories), the result outperforms the application of the two techniques in isolation.
From CNN to CNN + RNN: Adapting Visualization Techniques for Time-Series Anomaly Detection
Venue: arXiv cs.CV 2024
Year: 2024

Deep neural networks are highly effective in solving complex problems but are often viewed as "black boxes," limiting their adoption in contexts where transparency and explainability are essential. This lack of visibility raises ethical and legal concerns, particularly in critical areas like security, where automated decisions can have significant consequences. The General Data Protection Regulation (GDPR) underscores the importance of justifying these decisions. In this work, we explore visualization techniques to improve the understanding of anomaly detection models based on convolutional recurrent neural networks (CNN + RNN) with a TimeDistributed layer. Our model combines VGG19 for convolutional feature extraction and a GRU layer for sequential analysis of real-time video data. While suitable for temporal data, this structure complicates gradient propagation, as sequence elements are processed independently, dissociating temporal information. We adapt visualization techniques such as saliency maps and Grad-CAM to address these challenges. This article highlights the difficulties in visually interpreting video-based models and demonstrates how techniques for static images can be adapted to recurrent architectures, offering a transitional solution in the absence of dedicated methods.
NeuroFly: A framework for whole-brain single neuron reconstruction
Venue: arXiv cs.CV 2024
Year: 2024

Neurons, with their elongated, tree-like dendritic and axonal structures, enable efficient signal integration and long-range communication across brain regions. By reconstructing individual neurons' morphology, we can gain valuable insights into brain connectivity, revealing the structure basis of cognition, movement, and perception. Despite the accumulation of extensive 3D microscopic imaging data, progress has been considerably hindered by the absence of automated tools to streamline this process. Here we introduce NeuroFly, a validated framework for large-scale automatic single neuron reconstruction. This framework breaks down the process into three distinct stages: segmentation, connection, and proofreading. In the segmentation stage, we perform automatic segmentation followed by skeletonization to generate over-segmented neuronal fragments without branches. During the connection stage, we use a 3D image-based path following approach to extend each fragment and connect it with other fragments of the same neuron. Finally, human annotators are required only to proofread the few unresolved positions. The first two stages of our process are clearly defined computer vision problems, and we have trained robust baseline models to solve them. We validated NeuroFly's efficiency using in-house datasets that include a variety of challenging scenarios, such as dense arborizations, weak axons, images with contamination. We will release the datasets along with a suite of visualization and annotation tools for better reproducibility. Our goal is to foster collaboration among researchers to address the neuron reconstruction challenge, ultimately accelerating advancements in neuroscience research. The dataset and code are available at https://github.com/beanli161514/neurofly
MISGUIDE: Security-Aware Attack Analytics for Smart Grid Load Frequency Control
Venue: arXiv cs.CE 2024
Year: 2024

Incorporating advanced information and communication technologies into smart grids (SGs) offers substantial operational benefits while increasing vulnerability to cyber threats like false data injection (FDI) attacks. Current SG attack analysis tools predominantly employ formal methods or adversarial machine learning (ML) techniques with rule-based bad data detectors to analyze the attack space. However, these attack analytics either generate simplistic attack vectors detectable by the ML-based anomaly detection models (ADMs) or fail to identify critical attack vectors from complex controller dynamics in a feasible time. This paper introduces MISGUIDE, a novel defense-aware attack analytics designed to extract verifiable multi-time slot-based FDI attack vectors from complex SG load frequency control dynamics and ADMs, utilizing the Gurobi optimizer. MISGUIDE can identify optimal (maliciously triggering under/over frequency relays in minimal time) and stealthy attack vectors. Using real-world load data, we validate the MISGUIDE-identified attack vectors through real-time hardware-in-the-loop (OPALRT) simulations of the IEEE 39-bus system.