OpenRaiser/Intern-Atlas huggingface dataset:
Interactive tools for making temporally variable, multiple-attributes, and multiple-instances morphing accessible: Flexible manipulation of divergent speech instances for explorational research and education
Venue: arXiv cs.HC 2024
Year: 2024

We generalized a voice morphing algorithm capable of handling temporally variable, multiple-attributes, and multiple instances. The generalized morphing provides a new strategy for investigating speech diversity. However, excessive complexity and the difficulty of preparation have prevented researchers and students from enjoying its benefits. To address this issue, we introduced a set of interactive tools to make preparation and tests less cumbersome. These tools are integrated into our previously reported interactive tools as extensions. The introduction of the extended tools in lessons in graduate education was successful. Finally, we outline further extensions to explore excessively complex morphing parameter settings.
High-fidelity Endoscopic Image Synthesis by Utilizing Depth-guided Neural Surfaces
Venue: arXiv cs.CV 2024
Year: 2024

In surgical oncology, screening colonoscopy plays a pivotal role in providing diagnostic assistance, such as biopsy, and facilitating surgical navigation, particularly in polyp detection. Computer-assisted endoscopic surgery has recently gained attention and amalgamated various 3D computer vision techniques, including camera localization, depth estimation, surface reconstruction, etc. Neural Radiance Fields (NeRFs) and Neural Implicit Surfaces (NeuS) have emerged as promising methodologies for deriving accurate 3D surface models from sets of registered images, addressing the limitations of existing colon reconstruction approaches stemming from constrained camera movement. However, the inadequate tissue texture representation and confused scale problem in monocular colonoscopic image reconstruction still impede the progress of the final rendering results. In this paper, we introduce a novel method for colon section reconstruction by leveraging NeuS applied to endoscopic images, supplemented by a single frame of depth map. Notably, we pioneered the exploration of utilizing only one frame depth map in photorealistic reconstruction and neural rendering applications while this single depth map can be easily obtainable from other monocular depth estimation networks with an object scale. Through rigorous experimentation and validation on phantom imagery, our approach demonstrates exceptional accuracy in completely rendering colon sections, even capturing unseen portions of the surface. This breakthrough opens avenues for achieving stable and consistently scaled reconstructions, promising enhanced quality in cancer screening procedures and treatment interventions.
FSGe: A fast and strongly-coupled 3D fluid-solid-growth interaction method
Venue: arXiv cs.CE 2024
Year: 2024

Equilibrated fluid-solid-growth (FSGe) is a fast, open source, three-dimensional (3D) computational platform for simulating interactions between instantaneous hemodynamics and long-term vessel wall adaptation through mechanobiologically equilibrated growth and remodeling (G&R). Such models can capture evolving geometry, composition, and material properties in health and disease and following clinical interventions. In traditional G&R models, this feedback is modeled through highly simplified fluid solutions, neglecting local variations in blood pressure and wall shear stress (WSS). FSGe overcomes these inherent limitations by strongly coupling the 3D Navier-Stokes equations for blood flow with a 3D equilibrated constrained mixture model (CMMe) for vascular tissue G&R. CMMe allows one to predict long-term evolved mechanobiological equilibria from an original homeostatic state at a computational cost equivalent to that of a standard hyperelastic material model. In illustrative computational examples, we focus on the development of a stable aortic aneurysm in a mouse model to highlight key differences in growth patterns between FSGe and solid-only G&R models. We show that FSGe is especially important in blood vessels with asymmetric stimuli. Simulation results reveal greater local variation in fluid-derived WSS than in intramural stress (IMS). Thus, differences between FSGe and G&R models became more pronounced with the growing influence of WSS relative to pressure. Future applications in highly localized disease processes, such as for lesion formation in atherosclerosis, can now include spatial and temporal variations of WSS.
Cell Phone Image-Based Persian Rice Detection and Classification Using Deep Learning Techniques
Venue: arXiv cs.CV 2024
Year: 2024

This study introduces an innovative approach to classifying various types of Persian rice using image-based deep learning techniques, highlighting the practical application of everyday technology in food categorization. Recognizing the diversity of Persian rice and its culinary significance, we leveraged the capabilities of convolutional neural networks (CNNs), specifically by fine-tuning a ResNet model for accurate identification of different rice varieties and employing a U-Net architecture for precise segmentation of rice grains in bulk images. This dual-methodology framework allows for both individual grain classification and comprehensive analysis of bulk rice samples, addressing two crucial aspects of rice quality assessment. Utilizing images captured with consumer-grade cell phones reflects a realistic scenario in which individuals can leverage this technology for assistance with grocery shopping and meal preparation. The dataset, comprising various rice types photographed under natural conditions without professional lighting or equipment, presents a challenging yet practical classification problem. Our findings demonstrate the feasibility of using non-professional images for food classification and the potential of deep learning models, like ResNet and U-Net, to adapt to the nuances of everyday objects and textures. This study contributes to the field by providing insights into the applicability of image-based deep learning in daily life, specifically for enhancing consumer experiences and knowledge in food selection. Furthermore, it opens avenues for extending this approach to other food categories and practical applications, emphasizing the role of accessible technology in bridging the gap between sophisticated computational methods and everyday tasks.
Exploring AIGC Video Quality: A Focus on Visual Harmony, Video-Text Consistency and Domain Distribution Gap
Venue: arXiv cs.CV 2024
Year: 2024

The recent advancements in Text-to-Video Artificial Intelligence Generated Content (AIGC) have been remarkable. Compared with traditional videos, the assessment of AIGC videos encounters various challenges: visual inconsistency that defy common sense, discrepancies between content and the textual prompt, and distribution gap between various generative models, etc. Target at these challenges, in this work, we categorize the assessment of AIGC video quality into three dimensions: visual harmony, video-text consistency, and domain distribution gap. For each dimension, we design specific modules to provide a comprehensive quality assessment of AIGC videos. Furthermore, our research identifies significant variations in visual quality, fluidity, and style among videos generated by different text-to-video models. Predicting the source generative model can make the AIGC video features more discriminative, which enhances the quality assessment performance. The proposed method was used in the third-place winner of the NTIRE 2024 Quality Assessment for AI-Generated Content - Track 2 Video, demonstrating its effectiveness. Code will be available at https://github.com/Coobiw/TriVQA.
Opacity complexity of automatic sequences. The general case
Venue: arXiv cs.FL 2024
Year: 2024

In this work we introduce a new notion called opacity complexity to measure the complexity of automatic sequences. We study basic properties of this notion, and exhibit an algorithm to compute it. As applications, we compute the opacity complexity of some well-known automatic sequences, including in particular constant sequences, purely periodic sequences, the Thue-Morse sequence, the period-doubling sequence, the Golay-Shapiro(-Rudin) sequence, the paperfolding sequence, the Baum-Sweet sequence, the Tower of Hanoi sequence, and so on.
Beyond MMSE: Rank-1 Subspace Channel Estimator for Massive MIMO Systems
Venue: arXiv cs.IT 2024
Year: 2024

To glean the benefits offered by massive multi-input multi-output (MIMO) systems, channel state information must be accurately acquired. Despite the high accuracy, the computational complexity of classical linear minimum mean squared error (MMSE) estimator becomes prohibitively high in the context of massive MIMO, while the other low-complexity methods degrade the estimation accuracy seriously. In this paper, we develop a novel rank-1 subspace channel estimator to approximate the maximum likelihood (ML) estimator, which outperforms the linear MMSE estimator, but incurs a surprisingly low computational complexity. Our method first acquires the highly accurate angle-of-arrival (AoA) information via a constructed space-embedding matrix and the rank-1 subspace method. Then, it adopts the post-reception beamforming to acquire the unbiased estimate of channel gains. Furthermore, a fast method is designed to implement our new estimator. Theoretical analysis shows that the extra gain achieved by our method over the linear MMSE estimator grows according to the rule of O($\log_{10}M$), while its computational complexity is linearly scalable to the number of antennas $M$. Numerical simulations also validate the theoretical results. Our new method substantially extends the accuracy-complexity region and constitutes a promising channel estimation solution to the emerging massive MIMO communications.
The Branch Not Taken: Predicting Branching in Online Conversations
Venue: arXiv cs.CL 2024
Year: 2024

Multi-participant discussions tend to unfold in a tree structure rather than a chain structure. Branching may occur for multiple reasons -- from the asynchronous nature of online platforms to a conscious decision by an interlocutor to disengage with part of the conversation. Predicting branching and understanding the reasons for creating new branches is important for many downstream tasks such as summarization and thread disentanglement and may help develop online spaces that encourage users to engage in online discussions in more meaningful ways. In this work, we define the novel task of branch prediction and propose GLOBS (Global Branching Score) -- a deep neural network model for predicting branching. GLOBS is evaluated on three large discussion forums from Reddit, achieving significant improvements over an array of competitive baselines and demonstrating better transferability. We affirm that structural, temporal, and linguistic features contribute to GLOBS success and find that branching is associated with a greater number of conversation participants and tends to occur in earlier levels of the conversation tree. We publicly release GLOBS and our implementation of all baseline models to allow reproducibility and promote further research on this important task.
Parallel AIG Refactoring via Conflict Breaking
Venue: arXiv cs.DC 2024
Year: 2024

Algorithm parallelization to leverage multi-core platforms for improving the efficiency of Electronic Design Automation~(EDA) tools plays a significant role in enhancing the scalability of Integrated Circuit (IC) designs. Logic optimization is a key process in the EDA design flow to reduce the area and depth of the circuit graph by finding logically equivalent graphs for substitution, which is typically time-consuming. To address these challenges, in this paper, we first analyze two types of conflicts that need to be handled in the parallelization framework of refactoring And-Inverter Graph~(AIG). We then present a fine-grained parallel AIG refactoring method, which strikes a balance between the degree of parallelism and the conflicts encountered during the refactoring operations. Experiment results show that our parallel refactor is 28x averagely faster than the sequential algorithm on large benchmark tests with 64 physical CPU cores, and has comparable optimization quality.
Incorporating Different Verbal Cues to Improve Text-Based Computer-Delivered Health Messaging
Venue: arXiv cs.HC 2024
Year: 2024

The ubiquity of smartphones has led to an increase in on demand healthcare being supplied. For example, people can share their illness-related experiences with others similar to themselves, and healthcare experts can offer advice for better treatment and care for remediable, terminal and mental illnesses. As well as this human-to-human communication, there has been an increased use of human-to-computer digital health messaging, such as chatbots. These can prove advantageous as they offer synchronous and anonymous feedback without the need for a human conversational partner. However, there are many subtleties involved in human conversation that a computer agent may not properly exhibit. For example, there are various conversational styles, etiquettes, politeness strategies or empathic responses that need to be chosen appropriately for the conversation. Encouragingly, computers are social actors (CASA) posits that people apply the same social norms to computers as they would do to people. On from this, previous studies have focused on applying conversational strategies to computer agents to make them embody more favourable human characteristics. However, if a computer agent fails in this regard it can lead to negative reactions from users. Therefore, in this dissertation we describe a series of studies we carried out to lead to more effective human-to-computer digital health messaging. In our first study, we use the crowd [...] Our second study investigates the effect of a health chatbot's conversational style [...] In our final study, we investigate the format used by a chatbot when [...] In summary, we have researched how to create more effective digital health interventions starting from generating health messages, to choosing an appropriate formality of messaging, and finally to formatting messages which reference a user's previous utterances.
Performance Analysis for Deterministic System using Time Sensitive Network
Venue: arXiv cs.NI 2024
Year: 2024

Modern technology necessitates the use of dependable, fast, and inexpensive networks as the backbone for data transmission. Switched Ethernet coupled with the Time Sensitive Networking
MLP: Motion Label Prior for Temporal Sentence Localization in Untrimmed 3D Human Motions
Venue: arXiv cs.CV 2024
Year: 2024

In this paper, we address the unexplored question of temporal sentence localization in human motions (TSLM), aiming to locate a target moment from a 3D human motion that semantically corresponds to a text query. Considering that 3D human motions are captured using specialized motion capture devices, motions with only a few joints lack complex scene information like objects and lighting. Due to this character, motion data has low contextual richness and semantic ambiguity between frames, which limits the accuracy of predictions made by current video localization frameworks extended to TSLM to only a rough level. To refine this, we devise two novel label-prior-assisted training schemes: one embed prior knowledge of foreground and background to highlight the localization chances of target moments, and the other forces the originally rough predictions to overlap with the more accurate predictions obtained from the flipped start/end prior label sequences during recovery training. We show that injecting label-prior knowledge into the model is crucial for improving performance at high IoU. In our constructed TSLM benchmark, our model termed MLP achieves a recall of 44.13 at IoU@0.7 on the BABEL dataset and 71.17 on HumanML3D (Restore), outperforming prior works. Finally, we showcase the potential of our approach in corpus-level moment retrieval. Our source code is openly accessible at https://github.com/eanson023/mlp.
On de Bruijn Covering Sequences and Arrays
Venue: arXiv cs.IT 2024
Year: 2024

An $(m,n,R)$-de Bruijn covering array (dBCA) is a doubly periodic $M \times N$ array over an alphabet of size $q$ such that the set of all its $m \times n$ windows form a covering code with radius $R$. An upper bound of the smallest array area of an $(m,n,R)$-dBCA is provided using a probabilistic technique which is similar to the one that was used for an upper bound on the length of a de Bruijn covering sequence. A folding technique to construct a dBCA from a de Bruijn covering sequence or de Bruijn covering sequences code is presented. Several new constructions that yield shorter de Bruijn covering sequences and $(m,n,R)$-dBCAs with smaller areas are also provided. These constructions are mainly based on sequences derived from cyclic codes, self-dual sequences, primitive polynomials, an interleaving technique, folding, and mutual shifts of sequences with the same covering radius. Finally, constructions of de Bruijn covering sequences codes are also discussed.
Seamless Underwater Navigation with Limited Doppler Velocity Log Measurements
Venue: arXiv cs.RO 2024
Year: 2024

Autonomous Underwater Vehicles (AUVs) commonly utilize an inertial navigation system (INS) and a Doppler velocity log (DVL) for underwater navigation. To that end, their measurements are integrated through a nonlinear filter such as the extended Kalman filter (EKF). The DVL velocity vector estimate depends on retrieving reflections from the seabed, ensuring that at least three out of its four transmitted acoustic beams return successfully. When fewer than three beams are obtained, the DVL cannot provide a velocity update to bind the navigation solution drift. To cope with this challenge, in this paper, we propose a hybrid neural coupled (HNC) approach for seamless AUV navigation in situations of limited DVL measurements. First, we drive an approach to regress two or three missing DVL beams. Then, those beams, together with the measured beams, are incorporated into the EKF. We examined INS/DVL fusion both in loosely and tightly coupled approaches. Our method was trained and evaluated on recorded data from AUV experiments conducted in the Mediterranean Sea on two different occasions. The results illustrate that our proposed method outperforms the baseline loosely and tightly coupled model-based approaches by an average of 96.15%. It also demonstrates superior performance compared to a model-based beam estimator by an average of 12.41% in terms of velocity accuracy for scenarios involving two or three missing beams. Therefore, we demonstrate that our approach offers seamless AUV navigation in situations of limited beam measurements.
Efficient Digital Twin Data Processing for Low-Latency Multicast Short Video Streaming
Venue: arXiv cs.NI 2024
Year: 2024

In this paper, we propose a novel efficient digital twin (DT) data processing scheme to reduce service latency for multicast short video streaming. Particularly, DT is constructed to emulate and analyze user status for multicast group update and swipe feature abstraction. Then, a precise measurement model of DT data processing is developed to characterize the relationship among DT model size, user dynamics, and user clustering accuracy. A service latency model, consisting of DT data processing delay, video transcoding delay, and multicast transmission delay, is constructed by incorporating the impact of user clustering accuracy. Finally, a joint optimization problem of DT model size selection and bandwidth allocation is formulated to minimize the service latency. To efficiently solve this problem, a diffusion-based resource management algorithm is proposed, which utilizes the denoising technique to improve the action-generation process in the deep reinforcement learning algorithm. Simulation results based on the real-world dataset demonstrate that the proposed DT data processing scheme outperforms benchmark schemes in terms of service latency.
From LLM to NMT: Advancing Low-Resource Machine Translation with Claude
Venue: arXiv cs.CL 2024
Year: 2024

We show that Claude 3 Opus, a large language model (LLM) released by Anthropic in March 2024, exhibits stronger machine translation competence than other LLMs. Though we find evidence of data contamination with Claude on FLORES-200, we curate new benchmarks that corroborate the effectiveness of Claude for low-resource machine translation into English. We find that Claude has remarkable \textit{resource efficiency} -- the degree to which the quality of the translation model depends on a language pair's resource level. Finally, we show that advancements in LLM translation can be compressed into traditional neural machine translation (NMT) models. Using Claude to generate synthetic data, we demonstrate that knowledge distillation advances the state-of-the-art in Yoruba-English translation, meeting or surpassing strong baselines like NLLB-54B and Google Translate.
Swap It Like Its Hot: Segmentation-based spoof attacks on eye-tracking images
Venue: arXiv cs.CV 2024
Year: 2024

Video-based eye trackers capture the iris biometric and enable authentication to secure user identity. However, biometric authentication is susceptible to spoofing another user's identity through physical or digital manipulation. The current standard to identify physical spoofing attacks on eye-tracking sensors uses liveness detection. Liveness detection classifies gaze data as real or fake, which is sufficient to detect physical presentation attacks. However, such defenses cannot detect a spoofing attack when real eye image inputs are digitally manipulated to swap the iris pattern of another person. We propose IrisSwap as a novel attack on gaze-based liveness detection. IrisSwap allows attackers to segment and digitally swap in a victim's iris pattern to fool iris authentication. Both offline and online attacks produce gaze data that deceives the current state-of-the-art defense models at rates up to 58% and motivates the need to develop more advanced authentication methods for eye trackers.
Understanding the role of FFNs in driving multilingual behaviour in LLMs
Venue: arXiv cs.CL 2024
Year: 2024

Multilingualism in Large Language Models (LLMs) is an yet under-explored area. In this paper, we conduct an in-depth analysis of the multilingual capabilities of a family of a Large Language Model, examining its architecture, activation patterns, and processing mechanisms across languages. We introduce novel metrics to probe the model's multilingual behaviour at different layers and shed light on the impact of architectural choices on multilingual processing. Our findings reveal different patterns of multilinugal processing in the sublayers of Feed-Forward Networks of the models. Furthermore, we uncover the phenomenon of "over-layerization" in certain model configurations, where increasing layer depth without corresponding adjustments to other parameters may degrade model performance. Through comparisons within and across languages, we demonstrate the interplay between model architecture, layer depth, and multilingual processing capabilities of LLMs trained on multiple languages.
Beyond Personhood: Agency, Accountability, and the Limits of Anthropomorphic Ethical Analysis
Venue: arXiv cs.CY 2024
Year: 2024

What is agency, and why does it matter? In this work, we draw from the political science and philosophy literature and give two competing visions of what it means to be an (ethical) agent. The first view, which we term mechanistic, is commonly--and implicitly--assumed in AI research, yet it is a fundamentally limited means to understand the ethical characteristics of AI. Under the second view, which we term volitional, AI can no longer be considered an ethical agent. We discuss the implications of each of these views for two critical questions: first, what the ideal system ought to look like, and second, how accountability may be achieved. In light of this discussion, we ultimately argue that, in the context of ethically-significant behavior, AI should be viewed not as an agent but as the outcome of political processes.