OpenRaiser/Intern-Atlas huggingface dataset:
AniBalloons: Animated Chat Balloons as Affective Augmentation for Social Messaging and Chatbot Interaction
Venue: arXiv cs.HC 2024
Year: 2024

Despite being prominent and ubiquitous, message-based interaction is limited in nonverbally conveying emotions. Besides emoticons or stickers, messaging users continue seeking richer options for affective communication. Recent research explored using chat balloons' shape and color to communicate emotional states. However, little work explored whether and how chat-balloon animations could be designed to convey emotions. We present the design of AniBalloons, 30 chat-balloon animations conveying Joy, Anger, Sadness, Surprise, Fear, and Calmness. Using AniBalloons as a research means, we conducted three studies to assess the animations' affect recognizability and emotional properties (N = 40), and probe how animated chat balloons would influence communication experience in typical scenarios including instant messaging (N = 72) and chatbot service (N = 70). Our exploration contributes a set of chat-balloon animations to complement non-nonverbal affective communication for a range of message-based interfaces, and empirical insights into how animated chat balloons might mediate particular conversation experiences (e.g., perceived interpersonal closeness, or chatbot personality).
Dual Threats in RIS-Aided RF-UOWC Mixed Networks: Secrecy Performance Analysis under Simultaneous RF and UOWC Eavesdropping
Venue: arXiv cs.IT 2024
Year: 2024

In the dynamic realm of 6G technology, emphasizing security is essential, particularly for optimizing high-performance communication. A notable strategy involves the use of reconfigurable intelligent surfaces (RISs), an emerging and cost-efficient technology aimed at fortifying incoming signals, broadening coverage, and ultimately improving the overall performance of systems. In this paper, we introduce a comprehensive framework to analyze the secrecy performance of an RIS-assisted mixed radio frequency (RF) - underwater optical wireless communication (UOWC) network. Here, all the RF links undergo alpha-mu fading distribution, whereas the UOWC links experience a mixture of Exponential Generalized Gamma distribution. Specifically, we examine three potential eavesdropping situations: 1) eavesdropping on the RF link, 2) eavesdropping on the UOWC link, and 3) a simultaneous eavesdropping attack affecting both RF and UOWC links. To achieve this, we derive novel mathematical expressions such as average secrecy capacity, secrecy outage probability, strictly positive secrecy capacity, and effective secrecy throughput in closed form. Using these derived expressions, we carry out an investigation to assess the influences of fading parameters, pointing errors, receiver detection technique, underwater turbulence severity, and water salinity on the system. Furthermore, our study investigates the significance of RIS in improving secrecy performance due to the proposed model. To provide deeper insights, we also perform asymptotic analysis for the high signal-to-noise region. Finally, to verify our analytical results, we conduct Monte Carlo simulation using a computer-based technique.
VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agents
Venue: arXiv cs.AI 2024
Year: 2024

Large Multimodal Models (LMMs) have ushered in a new era in artificial intelligence, merging capabilities in both language and vision to form highly capable Visual Foundation Agents. These agents are postulated to excel across a myriad of tasks, potentially approaching general artificial intelligence. However, existing benchmarks fail to sufficiently challenge or showcase the full potential of LMMs in complex, real-world environments. To address this gap, we introduce VisualAgentBench (VAB), a comprehensive and pioneering benchmark specifically designed to train and evaluate LMMs as visual foundation agents across diverse scenarios, including Embodied, Graphical User Interface, and Visual Design, with tasks formulated to probe the depth of LMMs' understanding and interaction capabilities. Through rigorous testing across nine proprietary LMM APIs and eight open models, we demonstrate the considerable yet still developing agent capabilities of these models. Additionally, VAB constructs a trajectory training set constructed through hybrid methods including Program-based Solvers, LMM Agent Bootstrapping, and Human Demonstrations, promoting substantial performance improvements in LMMs through behavior cloning. Our work not only aims to benchmark existing models but also provides a solid foundation for future development into visual foundation agents. Code, train \& test data, and part of fine-tuned open LMMs are available at \url{https://github.com/THUDM/VisualAgentBench}.
Flow-Lenia.png: Evolving Multi-Scale Complexity by Means of Compression
Venue: arXiv cs.NE 2024
Year: 2024

We propose a fitness measure quantifying multi-scale complexity for cellular automaton states, using compressibility as a proxy for complexity. The use of compressibility is grounded in the concept of Kolmogorov complexity, which defines the complexity of an object by the size of its smallest representation. With this fitness function, we explore the complexity range accessible to the well-known Flow Lenia cellular automaton, using image compression algorithms to assess state compressibility. Using a Genetic Algorithm to evolve Flow Lenia patterns, we conduct experiments with two primary objectives: 1) generating patterns of specific complexity levels, and 2) exploring the extrema of Flow Lenia's complexity domain. Evolved patterns reflect the complexity targets, with higher complexity targets yielding more intricate patterns, consistent with human perceptions of complexity. This demonstrates that our fitness function can effectively evolve patterns that match specific complexity objectives within the bounds of the complexity range accessible to Flow Lenia under a given hyperparameter configuration.
FedRobo: Federated Learning Driven Autonomous Inter Robots Communication For Optimal Chemical Sprays
Venue: arXiv cs.LG 2024
Year: 2024

Federated Learning enables robots to learn from each other's experiences without relying on centralized data collection. Each robot independently maintains a model of crop conditions and chemical spray effectiveness, which is periodically shared with other robots in the fleet. A communication protocol is designed to optimize chemical spray applications by facilitating the exchange of information about crop conditions, weather, and other critical factors. The federated learning algorithm leverages this shared data to continuously refine the chemical spray strategy, reducing waste and improving crop yields. This approach has the potential to revolutionize the agriculture industry by offering a scalable and efficient solution for crop protection. However, significant challenges remain, including the development of a secure and robust communication protocol, the design of a federated learning algorithm that effectively integrates data from multiple sources, and ensuring the safety and reliability of autonomous robots. The proposed cluster-based federated learning approach also effectively reduces the computational load on the global server and minimizes communication overhead among clients.
Leading by the Nodes: A Survey of Film Industry Network Analysis and Datasets
Venue: arXiv cs.SI 2024
Year: 2024

This paper presents a comprehensive survey of network analysis research on the film industry, aiming to evaluate its emergence as a field of study and identify potential areas for further research. Many foundational network studies made use of the abundant data from the Internet Movie Database (IMDb) to test network methodologies. This survey focuses more specifically on examining research that employs network analysis to evaluate the film industry itself, revealing the social and business relationships involved in film production, distribution, and consumption. The paper adopts a classification approach based on node type and summarizes the key contributions in relation to each. The review provides insights into the structure and interconnectedness of the field, highlighting clusters of debates and shedding light on the areas in need of further theoretical and methodological development. In addition, this survey contributes to understanding film industry network analysis and informs researchers interested in network methods within the film industry and related cultural sectors.
Synthetic Photography Detection: A Visual Guidance for Identifying Synthetic Images Created by AI
Venue: arXiv cs.CY 2024
Year: 2024

Artificial Intelligence (AI) tools have become incredibly powerful in generating synthetic images. Of particular concern are generated images that resemble photographs as they aspire to represent real world events. Synthetic photographs may be used maliciously by a broad range of threat actors, from scammers to nation-state actors, to deceive, defraud, and mislead people. Mitigating this threat usually involves answering a basic analytic question: Is the photograph real or synthetic? To address this, we have examined the capabilities of recent generative diffusion models and have focused on their flaws: visible artifacts in generated images which reveal their synthetic origin to the trained eye. We categorize these artifacts, provide examples, discuss the challenges in detecting them, suggest practical applications of our work, and outline future research directions.
Evaluating LLMs on Entity Disambiguation in Tables
Venue: arXiv cs.CL 2024
Year: 2024

Tables are crucial containers of information, but understanding their meaning may be challenging. Over the years, there has been a surge in interest in data-driven approaches based on deep learning that have increasingly been combined with heuristic-based ones. In the last period, the advent of \acf{llms} has led to a new category of approaches for table annotation. However, these approaches have not been consistently evaluated on a common ground, making evaluation and comparison difficult. This work proposes an extensive evaluation of four STI SOTA approaches: Alligator (formerly s-elbat), Dagobah, TURL, and TableLlama; the first two belong to the family of heuristic-based algorithms, while the others are respectively encoder-only and decoder-only Large Language Models (LLMs). We also include in the evaluation both GPT-4o and GPT-4o-mini, since they excel in various public benchmarks. The primary objective is to measure the ability of these approaches to solve the entity disambiguation task with respect to both the performance achieved on a common-ground evaluation setting and the computational and cost requirements involved, with the ultimate aim of charting new research paths in the field.
Addressing the Unforeseen Harms of Technology CCC Whitepaper
Venue: arXiv cs.CY 2024
Year: 2024

Recent years have seen increased awareness of the potential significant impacts of computing technologies, both positive and negative. This whitepaper explores how to address possible harmful consequences of computing technologies that might be difficult to anticipate, and thereby mitigate or address. It starts from the assumption that very few harms due to technology are intentional or deliberate; rather, the vast majority result from failure to recognize and respond to them prior to deployment. Nonetheless, there are concrete steps that can be taken to address the difficult problem of anticipating and responding to potential harms from new technologies.
Statistical Quality Comparison of the Bitstrings Generated by a Physical Unclonable Function across Xilinx, Altera and Microsemi Devices
Venue: arXiv cs.CR 2024
Year: 2024

Entropy or randomness represents a foundational security property in security-related operations, such as key generation. Key generation in turn is central to security protocols such as authentication and encryption. Physical unclonable functions (PUF) are hardware-based primitives that can serve as key generation engines in modern microelectronic devices and applications. PUFs derive entropy from manufacturing variations that exist naturally within and across otherwise identical copies of a device. However, the levels of random variations that represent entropy, which are strongly correlated to the quality of the PUF-generated bitstrings, vary from one manufacturer to another. In this paper, we evaluate entropy across a set of devices manufactured by three mainstream FPGA vendors, Xilinx, Altera and Microsemi. The devices selected for evaluation are considered low-end commercial devices to make the analysis relevant to IoT applications. The SiRF PUF is used in the evaluation, and is constructed nearly identically across the three vendor devices, setting aside minor differences that exist in certain logic element primitives used within the PUF architecture, and which have only a minor impact on our comparative analysis. The SiRF PUF uses a high-resolution time-to-digital converter (TDC) crafted from high-speed carry-chain logic embedded within each device to measure path delays in an engineered netlist of logic gates as a source of entropy. Therefore, our analysis includes an evaluation of actual path delay variation as it exists across the three device classes, as well as a statistical evaluation of the PUF-generated bitstrings. A reliablity analysis is also provided using data collected in industrial-standard temperature experiments to round out the evaluation of important statistical properties of the PUF.
Quasi-Monte Carlo Beyond Hardy-Krause
Venue: arXiv cs.DS 2024
Year: 2024

The classical approaches to numerically integrating a function $f$ are Monte Carlo (MC) and quasi-Monte Carlo (QMC) methods. MC methods use random samples to evaluate $f$ and have error $O(σ(f)/\sqrt{n})$, where $σ(f)$ is the standard deviation of $f$. QMC methods are based on evaluating $f$ at explicit point sets with low discrepancy, and as given by the classical Koksma-Hlawka inequality, they have error $\widetilde{O}(σ_{\mathsf{HK}}(f)/n)$, where $σ_{\mathsf{HK}}(f)$ is the variation of $f$ in the sense of Hardy and Krause. These two methods have distinctive advantages and shortcomings, and a fundamental question is to find a method that combines the advantages of both. In this work, we give a simple randomized algorithm that produces QMC point sets with the following desirable features: (1) It achieves substantially better error than given by the classical Koksma-Hlawka inequality. In particular, it has error $\widetilde{O}(σ_{\mathsf{SO}}(f)/n)$, where $σ_{\mathsf{SO}}(f)$ is a new measure of variation that we introduce, which is substantially smaller than the Hardy-Krause variation. (2) The algorithm only requires random samples from the underlying distribution, which makes it as flexible as MC. (3) It automatically achieves the best of both MC and QMC (and the above improvement over Hardy-Krause variation) in an optimal way. (4) The algorithm is extremely efficient, with an amortized $\widetilde{O}(1)$ runtime per sample. Our method is based on the classical transference principle in geometric discrepancy, combined with recent algorithmic innovations in combinatorial discrepancy that besides producing low-discrepancy colorings, also guarantee certain subgaussian properties. This allows us to bypass several limitations of previous works in bridging the gap between MC and QMC methods and go beyond the Hardy-Krause variation.
Benchmarking tree species classification from proximally-sensed laser scanning data: introducing the FOR-species20K dataset
Venue: arXiv cs.CV 2024
Year: 2024

Proximally-sensed laser scanning offers significant potential for automated forest data capture, but challenges remain in automatically identifying tree species without additional ground data. Deep learning (DL) shows promise for automation, yet progress is slowed by the lack of large, diverse, openly available labeled datasets of single tree point clouds. This has impacted the robustness of DL models and the ability to establish best practices for species classification. To overcome these challenges, the FOR-species20K benchmark dataset was created, comprising over 20,000 tree point clouds from 33 species, captured using terrestrial (TLS), mobile (MLS), and drone laser scanning (ULS) across various European forests, with some data from other regions. This dataset enables the benchmarking of DL models for tree species classification, including both point cloud-based (PointNet++, MinkNet, MLP-Mixer, DGCNNs) and multi-view image-based methods (SimpleView, DetailView, YOLOv5). 2D image-based models generally performed better (average OA = 0.77) than 3D point cloud-based models (average OA = 0.72), with consistent results across different scanning platforms and sensors. The top model, DetailView, was particularly robust, handling data imbalances well and generalizing effectively across tree sizes. The FOR-species20K dataset, available at https://zenodo.org/records/13255198, is a key resource for developing and benchmarking DL models for tree species classification using laser scanning data, providing a foundation for future advancements in the field.
Introducing the NewsPaLM MBR and QE Dataset: LLM-Generated High-Quality Parallel Data Outperforms Traditional Web-Crawled Data
Venue: arXiv cs.CL 2024
Year: 2024

Recent research in neural machine translation (NMT) has shown that training on high-quality machine-generated data can outperform training on human-generated data. This work accompanies the first-ever release of a LLM-generated, MBR-decoded and QE-reranked dataset with both sentence-level and multi-sentence examples. We perform extensive experiments to demonstrate the quality of our dataset in terms of its downstream impact on NMT model performance. We find that training from scratch on our (machine-generated) dataset outperforms training on the (web-crawled) WMT'23 training dataset (which is 300 times larger), and also outperforms training on the top-quality subset of the WMT'23 training dataset. We also find that performing self-distillation by finetuning the LLM which generated this dataset outperforms the LLM's strong few-shot baseline. These findings corroborate the quality of our dataset, and demonstrate the value of high-quality machine-generated data in improving performance of NMT models.
Understanding Power Consumption Metric on Heterogeneous Memory Systems
Venue: arXiv cs.PF 2024
Year: 2024

Contemporary memory systems contain a variety of memory types, each possessing distinct characteristics. This trend empowers applications to opt for memory types aligning with developer's desired behavior. As a result, developers gain flexibility to tailor their applications to specific needs, factoring in attributes like latency, bandwidth, and power consumption. Our research centers on the aspect of power consumption within memory systems. We introduce an approach that equips developers with comprehensive insights into the power consumption of individual memory types. Additionally, we propose an ordered hierarchy of memory types. Through this methodology, developers can make informed decisions for efficient memory usage aligned with their unique requirements.
ActiveNeRF: Learning Accurate 3D Geometry by Active Pattern Projection
Venue: arXiv cs.CV 2024
Year: 2024

NeRFs have achieved incredible success in novel view synthesis. However, the accuracy of the implicit geometry is unsatisfactory because the passive static environmental illumination has low spatial frequency and cannot provide enough information for accurate geometry reconstruction. In this work, we propose ActiveNeRF, a 3D geometry reconstruction framework, which improves the geometry quality of NeRF by actively projecting patterns of high spatial frequency onto the scene using a projector which has a constant relative pose to the camera. We design a learnable active pattern rendering pipeline which jointly learns the scene geometry and the active pattern. We find that, by adding the active pattern and imposing its consistency across different views, our proposed method outperforms state of the art geometry reconstruction methods qualitatively and quantitatively in both simulation and real experiments. Code is avaliable at https://github.com/hcp16/active_nerf
ActPrompt: In-Domain Feature Adaptation via Action Cues for Video Temporal Grounding
Venue: arXiv cs.CV 2024
Year: 2024

Video temporal grounding is an emerging topic aiming to identify specific clips within videos. In addition to pre-trained video models, contemporary methods utilize pre-trained vision-language models (VLM) to capture detailed characteristics of diverse scenes and objects from video frames. However, as pre-trained on images, VLM may struggle to distinguish action-sensitive patterns from static objects, making it necessary to adapt them to specific data domains for effective feature representation over temporal grounding. We address two primary challenges to achieve this goal. Specifically, to mitigate high adaptation costs, we propose an efficient preliminary in-domain fine-tuning paradigm for feature adaptation, where downstream-adaptive features are learned through several pretext tasks. Furthermore, to integrate action-sensitive information into VLM, we introduce Action-Cue-Injected Temporal Prompt Learning (ActPrompt), which injects action cues into the image encoder of VLM for better discovering action-sensitive patterns. Extensive experiments demonstrate that ActPrompt is an off-the-shelf training framework that can be effectively applied to various SOTA methods, resulting in notable improvements. The complete code used in this study is provided in the supplementary materials.
DePatch: Towards Robust Adversarial Patch for Evading Person Detectors in the Real World
Venue: arXiv cs.CV 2024
Year: 2024

Recent years have seen an increasing interest in physical adversarial attacks, which aim to craft deployable patterns for deceiving deep neural networks, especially for person detectors. However, the adversarial patterns of existing patch-based attacks heavily suffer from the self-coupling issue, where a degradation, caused by physical transformations, in any small patch segment can result in a complete adversarial dysfunction, leading to poor robustness in the complex real world. Upon this observation, we introduce the Decoupled adversarial Patch (DePatch) attack to address the self-coupling issue of adversarial patches. Specifically, we divide the adversarial patch into block-wise segments, and reduce the inter-dependency among these segments through randomly erasing out some segments during the optimization. We further introduce a border shifting operation and a progressive decoupling strategy to improve the overall attack capabilities. Extensive experiments demonstrate the superior performance of our method over other physical adversarial attacks, especially in the real world.