OpenRaiser/Intern-Atlas huggingface dataset:
Guided Diffusion for Fast Inverse Design of Density-based Mechanical Metamaterials
Venue: arXiv cs.CE 2024
Year: 2024

Mechanical metamaterial is a synthetic material that can possess extraordinary physical characteristics, such as abnormal elasticity, stiffness, and stability, by carefully designing its internal structure. To make metamaterials contain delicate local structures with unique mechanical properties, it is a potential method to represent them through high-resolution voxels. However, it brings a substantial computational burden. To this end, this paper proposes a fast inverse design method, whose core is an advanced deep generative AI algorithm, to generate voxel-based mechanical metamaterials. Specifically, we use the self-conditioned diffusion model, capable of generating a microstructure with a resolution of $128^3$ to approach the specified homogenized tensor matrix in just 3 seconds. Accordingly, this rapid reverse design tool facilitates the exploration of extreme metamaterials, the sequence interpolation in metamaterials, and the generation of diverse microstructures for multi-scale design. This flexible and adaptive generative tool is of great value in structural engineering or other mechanical systems and can stimulate more subsequent research.
Securing the Invisible Thread: A Comprehensive Analysis of BLE Tracker Security in Apple AirTags and Samsung SmartTags
Venue: arXiv cs.CR 2024
Year: 2024

This study presents an in-depth analysis of the security landscape in Bluetooth Low Energy (BLE) tracking systems, with a particular emphasis on Apple AirTags and Samsung SmartTags, including their cryptographic frameworks. Our investigation traverses a wide spectrum of attack vectors such as physical tampering, firmware exploitation, signal spoofing, eavesdropping, jamming, app security flaws, Bluetooth security weaknesses, location spoofing, threats to owner devices, and cloud-related vulnerabilities. Moreover, we delve into the security implications of the cryptographic methods utilized in these systems. Our findings reveal that while BLE trackers like AirTags and SmartTags offer substantial utility, they also pose significant security risks. Notably, Apple's approach, which prioritizes user privacy by removing intermediaries, inadvertently leads to device authentication challenges, evidenced by successful AirTag spoofing instances. Conversely, Samsung SmartTags, designed to thwart beacon spoofing, raise critical concerns about cloud security and user privacy. Our analysis also highlights the constraints faced by these devices due to their design focus on battery life conservation, particularly the absence of secure boot processes, which leaves them susceptible to OS modification and a range of potential attacks. The paper concludes with insights into the anticipated evolution of these tracking systems. We predict that future enhancements will likely focus on bolstering security features, especially as these devices become increasingly integrated into the broader IoT ecosystem and face evolving privacy regulations. This shift is imperative to address the intricate balance between functionality and security in next-generation BLE tracking systems.
Deep Learning Based Adaptive Joint mmWave Beam Alignment
Venue: arXiv cs.IT 2024
Year: 2024

The challenging propagation environment, combined with the hardware limitations of mmWave systems, gives rise to the need for accurate initial access beam alignment strategies with low latency and high achievable beamforming gain. Much of the recent work in this area either focuses on one-sided beam alignment, or, joint beam alignment methods where both sides of the link perform a sequence of fixed channel probing steps. Codebook-based non-adaptive beam alignment schemes have the potential to allow multiple user equipment (UE) to perform initial access beam alignment in parallel whereas adaptive schemes are favourable in achievable beamforming gain. This work introduces a novel deep learning based joint beam alignment scheme that aims to combine the benefits of adaptive, codebook-free beam alignment at the UE side with the advantages of a codebook-sweep based scheme at the base station. The proposed end-to-end trainable scheme is compatible with current cellular standard signaling and can be readily integrated into the standard without requiring significant changes to it. Extensive simulations demonstrate superior performance of the proposed approach over purely codebook-based ones.
Evaluation of General Large Language Models in Contextually Assessing Semantic Concepts Extracted from Adult Critical Care Electronic Health Record Notes
Venue: arXiv cs.CL 2024
Year: 2024

The field of healthcare has increasingly turned its focus towards Large Language Models (LLMs) due to their remarkable performance. However, their performance in actual clinical applications has been underexplored. Traditional evaluations based on question-answering tasks don't fully capture the nuanced contexts. This gap highlights the need for more in-depth and practical assessments of LLMs in real-world healthcare settings. Objective: We sought to evaluate the performance of LLMs in the complex clinical context of adult critical care medicine using systematic and comprehensible analytic methods, including clinician annotation and adjudication. Methods: We investigated the performance of three general LLMs in understanding and processing real-world clinical notes. Concepts from 150 clinical notes were identified by MetaMap and then labeled by 9 clinicians. Each LLM's proficiency was evaluated by identifying the temporality and negation of these concepts using different prompts for an in-depth analysis. Results: GPT-4 showed overall superior performance compared to other LLMs. In contrast, both GPT-3.5 and text-davinci-003 exhibit enhanced performance when the appropriate prompting strategies are employed. The GPT family models have demonstrated considerable efficiency, evidenced by their cost-effectiveness and time-saving capabilities. Conclusion: A comprehensive qualitative performance evaluation framework for LLMs is developed and operationalized. This framework goes beyond singular performance aspects. With expert annotations, this methodology not only validates LLMs' capabilities in processing complex medical data but also establishes a benchmark for future LLM evaluations across specialized domains.
Intermittent Connectivity Maintenance With Heterogeneous Robots
Venue: arXiv cs.RO 2024
Year: 2024

We consider a scenario of cooperative task servicing, with a team of heterogeneous robots with different maximum speeds and communication radii, in charge of keeping the network intermittently connected. We abstract the task locations into a $1D$ cycle graph that is traversed by the communicating robots, and we discuss intermittent communication strategies so that each task location is periodically visited, with a worst--case revisiting time. Robots move forward and backward along the cycle graph, exchanging data with their previous and next neighbors when they meet, and updating their region boundaries. Asymptotically, each robot is in charge of a region of the cycle graph, depending on its capabilities. The method is distributed, and robots only exchange data when they meet.
Employing polyhedral methods to optimize stencils on FPGAs with stencil-specific caches, data reuse, and wide data bursts
Venue: arXiv cs.PL 2024
Year: 2024

It is well known that to accelerate stencil codes on CPUs or GPUs and to exploit hardware caches and their lines optimizers must find spatial and temporal locality of array accesses to harvest data-reuse opportunities. On FPGAs there is the burden that there are no built-in caches (or only pre-built hardware descriptions for cache blocks that are inefficient for stencil codes). But this paper demonstrates that this lack is also a chance as polyhedral methods can be used to generate stencil-specific cache-structures of the right sizes on the FPGA and to fill and flush them efficiently with wide bursts during stencil execution. The paper shows how to derive the appropriate directives and code restructurings from stencil codes so that the FPGA compiler generates fast stencil hardware. Switching on our optimization improves the runtime of a set of 10 stencils by between 43x and 156x.
The Definitive Guide to Policy Gradients in Deep Reinforcement Learning: Theory, Algorithms and Implementations
Venue: arXiv cs.LG 2024
Year: 2024

In recent years, various powerful policy gradient algorithms have been proposed in deep reinforcement learning. While all these algorithms build on the Policy Gradient Theorem, the specific design choices differ significantly across algorithms. We provide a holistic overview of on-policy policy gradient algorithms to facilitate the understanding of both their theoretical foundations and their practical implementations. In this overview, we include a detailed proof of the continuous version of the Policy Gradient Theorem, convergence results and a comprehensive discussion of practical algorithms. We compare the most prominent algorithms on continuous control environments and provide insights on the benefits of regularization. All code is available at https://github.com/Matt00n/PolicyGradientsJax.
Value-Driven Mixed-Precision Quantization for Patch-Based Inference on Microcontrollers
Venue: arXiv cs.CV 2024
Year: 2024

Deploying neural networks on microcontroller units (MCUs) presents substantial challenges due to their constrained computation and memory resources. Previous researches have explored patch-based inference as a strategy to conserve memory without sacrificing model accuracy. However, this technique suffers from severe redundant computation overhead, leading to a substantial increase in execution latency. A feasible solution to address this issue is mixed-precision quantization, but it faces the challenges of accuracy degradation and a time-consuming search time. In this paper, we propose QuantMCU, a novel patch-based inference method that utilizes value-driven mixed-precision quantization to reduce redundant computation. We first utilize value-driven patch classification (VDPC) to maintain the model accuracy. VDPC classifies patches into two classes based on whether they contain outlier values. For patches containing outlier values, we apply 8-bit quantization to the feature maps on the dataflow branches that follow. In addition, for patches without outlier values, we utilize value-driven quantization search (VDQS) on the feature maps of their following dataflow branches to reduce search time. Specifically, VDQS introduces a novel quantization search metric that takes into account both computation and accuracy, and it employs entropy as an accuracy representation to avoid additional training. VDQS also adopts an iterative approach to determine the bitwidth of each feature map to further accelerate the search process. Experimental results on real-world MCU devices show that QuantMCU can reduce computation by 2.2x on average while maintaining comparable model accuracy compared to the state-of-the-art patch-based inference methods.
Diffuse to Choose: Enriching Image Conditioned Inpainting in Latent Diffusion Models for Virtual Try-All
Venue: arXiv cs.CV 2024
Year: 2024

As online shopping is growing, the ability for buyers to virtually visualize products in their settings-a phenomenon we define as "Virtual Try-All"-has become crucial. Recent diffusion models inherently contain a world model, rendering them suitable for this task within an inpainting context. However, traditional image-conditioned diffusion models often fail to capture the fine-grained details of products. In contrast, personalization-driven models such as DreamPaint are good at preserving the item's details but they are not optimized for real-time applications. We present "Diffuse to Choose," a novel diffusion-based image-conditioned inpainting model that efficiently balances fast inference with the retention of high-fidelity details in a given reference item while ensuring accurate semantic manipulations in the given scene content. Our approach is based on incorporating fine-grained features from the reference image directly into the latent feature maps of the main diffusion model, alongside with a perceptual loss to further preserve the reference item's details. We conduct extensive testing on both in-house and publicly available datasets, and show that Diffuse to Choose is superior to existing zero-shot diffusion inpainting methods as well as few-shot diffusion personalization algorithms like DreamPaint.
Who Changed the Destiny of Rural Students, and How?: Unpacking ICT-Mediated Remote Education in Rural China
Venue: arXiv cs.CY 2024
Year: 2024

The proliferation of Information and Communication Technologies (ICTs) has shown great promise in addressing educational challenges facing rural areas. However, the complex rural context poses significant challenges to the effective utilization of these technologies. This paper examines the empirical integration of live-streaming-based remote classrooms (LSRC) through a qualitative study in rural China. Our findings suggest that while LSRC enables rural students equal access to high-quality educational resources, its practical integration faces numerous challenges. In particular, we emphasize the crucial role of local teachers in addressing these challenges, ultimately achieving the desired improvement of students' learning outcomes. We also examine the impact of LSRC on the original rural education ecosystem. Building upon our findings, we call for a reconsideration of interaction paradigms and evaluation systems of ICT-mediated rural education, emphasizing the significance of rural teachers. We conclude by discussing the implications for future ICT-mediated technology interventions in rural settings.
Robustness in Fairness against Edge-level Perturbations in GNN-based Recommendation
Venue: arXiv cs.IR 2024
Year: 2024

Efforts in the recommendation community are shifting from the sole emphasis on utility to considering beyond-utility factors, such as fairness and robustness. Robustness of recommendation models is typically linked to their ability to maintain the original utility when subjected to attacks. Limited research has explored the robustness of a recommendation model in terms of fairness, e.g., the parity in performance across groups, under attack scenarios. In this paper, we aim to assess the robustness of graph-based recommender systems concerning fairness, when exposed to attacks based on edge-level perturbations. To this end, we considered four different fairness operationalizations, including both consumer and provider perspectives. Experiments on three datasets shed light on the impact of perturbations on the targeted fairness notion, uncovering key shortcomings in existing evaluation protocols for robustness. As an example, we observed perturbations affect consumer fairness on a higher extent than provider fairness, with alarming unfairness for the former. Source code: https://github.com/jackmedda/CPFairRobust
Enumerating the k-fold configurations in multi-class classification problems
Venue: arXiv cs.LG 2024
Year: 2024

K-fold cross-validation is a widely used tool for assessing classifier performance. The reproducibility crisis faced by artificial intelligence partly results from the irreproducibility of reported k-fold cross-validation-based performance scores. Recently, we introduced numerical techniques to test the consistency of claimed performance scores and experimental setups. In a crucial use case, the method relies on the combinatorial enumeration of all k-fold configurations, for which we proposed an algorithm in the binary classification case.
Shell topology optimization based on level set method
Venue: arXiv cs.CE 2024
Year: 2024

This paper proposes a level set-based method for optimizing shell structures with large design changes in shape and topology. Conventional shell optimization methods, whether parametric or nonparametric, often only allow limited design changes in shape. In the proposed method, the shell structure is defined as the isosurface of a level set function. The level set function is iteratively updated based on the shape sensitivity on the surface mesh. Therefore, the proposed method can represent an arbitrary manifold surface while dealing with topological changes, for example, from a spherical surface to a toroidal surface. We applied the proposed method to the mean compliance minimization problems of 3D shell structural designs for dome, bending plate and cantilever beam examples to demonstrate its efficacy of the proposed method.
Knowledge Guided Entity-aware Video Captioning and A Basketball Benchmark
Venue: arXiv cs.CV 2024
Year: 2024

Despite the recent emergence of video captioning models, how to generate the text description with specific entity names and fine-grained actions is far from being solved, which however has great applications such as basketball live text broadcast. In this paper, a new multimodal knowledge graph supported basketball benchmark for video captioning is proposed. Specifically, we construct a multimodal basketball game knowledge graph (KG_NBA_2022) to provide additional knowledge beyond videos. Then, a multimodal basketball game video captioning (VC_NBA_2022) dataset that contains 9 types of fine-grained shooting events and 286 players' knowledge (i.e., images and names) is constructed based on KG_NBA_2022. We develop a knowledge guided entity-aware video captioning network (KEANet) based on a candidate player list in encoder-decoder form for basketball live text broadcast. The temporal contextual information in video is encoded by introducing the bi-directional GRU (Bi-GRU) module. And the entity-aware module is designed to model the relationships among the players and highlight the key players. Extensive experiments on multiple sports benchmarks demonstrate that KEANet effectively leverages extera knowledge and outperforms advanced video captioning models. The proposed dataset and corresponding codes will be publicly available soon
Text to speech synthesis
Venue: arXiv cs.SE 2024
Year: 2024

Text-to-speech (TTS) synthesis is a technology that converts written text into spoken words, enabling a natural and accessible means of communication. This abstract explores the key aspects of TTS synthesis, encompassing its underlying technologies, applications, and implications for various sectors. The technology utilizes advanced algorithms and linguistic models to convert textual information into life like speech, allowing for enhanced user experiences in diverse contexts such as accessibility tools, navigation systems, and virtual assistants. The abstract delves into the challenges and advancements in TTS synthesis, including considerations for naturalness, multilingual support, and emotional expression in synthesized speech.
No More Distractions: an Adaptive Up-Sampling Algorithm to Reduce Data Artifacts
Venue: arXiv cs.CL 2024
Year: 2024

Researchers recently found out that sometimes language models achieve high accuracy on benchmark data set, but they can not generalize very well with even little changes to the original data set. This is sometimes due to data artifacts, model is learning the spurious correlation between tokens and labels, instead of the semantics and logic. In this work, we analyzed SNLI data and visualized such spurious correlations. We proposed an adaptive up-sampling algorithm to correct the data artifacts, which is simple and effective, and does not need human edits or annotation. We did an experiment applying the algorithm to fix the data artifacts in SNLI data and the model trained with corrected data performed significantly better than the model trained with raw SNLI data, overall, as well as on the subset we corrected.
Spectral Clustering for Discrete Distributions
Venue: arXiv cs.LG 2024
Year: 2024

The discrete distribution is often used to describe complex instances in machine learning, such as images, sequences, and documents. Traditionally, clustering of discrete distributions (D2C) has been approached using Wasserstein barycenter methods. These methods operate under the assumption that clusters can be well-represented by barycenters, which is seldom true in many real-world applications. Additionally, these methods are not scalable for large datasets due to the high computational cost of calculating Wasserstein barycenters. In this work, we explore the feasibility of using spectral clustering combined with distribution affinity measures (e.g., maximum mean discrepancy and Wasserstein distance) to cluster discrete distributions. We demonstrate that these methods can be more accurate and efficient than barycenter methods. To further enhance scalability, we propose using linear optimal transport to construct affinity matrices efficiently for large datasets. We provide theoretical guarantees for the success of our methods in clustering distributions. Experiments on both synthetic and real data show that our methods outperform existing baselines.
Image based Crop Monitoring Technologies in Protected Horticulture: A Review
Venue: arXiv cs.CE 2024
Year: 2024

Future food security is a major concern of the 21st century with the growing global population and climate changes. In addressing these challenges, protected cropping ensures food production year-round and increases crop production per land area by controlling environment conditions. Maintaining the growth and health of crops in these facilities is essential to ensure optimum food production. However, this is a laborious work and is currently done manually. Image-based non-destructive plant phenotyping is an emerging research area that reduces the skilled labour cost while enhancing the monitoring of crop growth, health, and identifying phenotype-genotype relations for plant breeding. With the proliferations of protected infrastructures and targeted plants, different technologies and sensor setups are needed for image-based crop monitoring. Conveyor-type plant-to-sensor systems, bench-top or gantry-based systems are commonly found in research facilities focussing on phenotyping of small, relatively short, or movable model plants. This review examines the literature on crop monitoring and phenotyping platforms in both field and protected facilities and explains different camera technologies and their ability to extract different plant traits. The review highlights the future research directions of image-based monitoring of commercial scale protected crops where crops can be relatively tall or vertically supported under semi controlled environments, which presents new challenges and is rarely covered in the literature.
AM-SORT: Adaptable Motion Predictor with Historical Trajectory Embedding for Multi-Object Tracking
Venue: arXiv cs.CV 2024
Year: 2024

Many multi-object tracking (MOT) approaches, which employ the Kalman Filter as a motion predictor, assume constant velocity and Gaussian-distributed filtering noises. These assumptions render the Kalman Filter-based trackers effective in linear motion scenarios. However, these linear assumptions serve as a key limitation when estimating future object locations within scenarios involving non-linear motion and occlusions. To address this issue, we propose a motion-based MOT approach with an adaptable motion predictor, called AM-SORT, which adapts to estimate non-linear uncertainties. AM-SORT is a novel extension of the SORT-series trackers that supersedes the Kalman Filter with the transformer architecture as a motion predictor. We introduce a historical trajectory embedding that empowers the transformer to extract spatio-temporal features from a sequence of bounding boxes. AM-SORT achieves competitive performance compared to state-of-the-art trackers on DanceTrack, with 56.3 IDF1 and 55.6 HOTA. We conduct extensive experiments to demonstrate the effectiveness of our method in predicting non-linear movement under occlusions.