OpenRaiser/Intern-Atlas huggingface dataset:
Accelerating Deep Neural Networks via Semi-Structured Activation Sparsity
Venue: arXiv cs.CV 2023
Year: 2023

The demand for efficient processing of deep neural networks (DNNs) on embedded devices is a significant challenge limiting their deployment. Exploiting sparsity in the network's feature maps is one of the ways to reduce its inference latency. It is known that unstructured sparsity results in lower accuracy degradation with respect to structured sparsity but the former needs extensive inference engine changes to get latency benefits. To tackle this challenge, we propose a solution to induce semi-structured activation sparsity exploitable through minor runtime modifications. To attain high speedup levels at inference time, we design a sparse training procedure with awareness of the final position of the activations while computing the General Matrix Multiplication (GEMM). We extensively evaluate the proposed solution across various models for image classification and object detection tasks. Remarkably, our approach yields a speed improvement of $1.25 \times$ with a minimal accuracy drop of $1.1\%$ for the ResNet18 model on the ImageNet dataset. Furthermore, when combined with a state-of-the-art structured pruning method, the resulting models provide a good latency-accuracy trade-off, outperforming models that solely employ structured pruning techniques.
MCQUIC: Multicast and unicast in a single transport protocol
Venue: arXiv cs.NI 2023
Year: 2023

Multicast enables efficient one-to-many communications. Several applications benefit from its scalability properties, e.g., live-streaming and large-scale software updates. Historically, multicast applications have used specialized transport protocols. The flexibility of the recently standardized QUIC protocol opens the possibility of providing both unicast and multicast services to applications with a single transport protocol. We present MCQUIC, an extended version of the QUIC protocol that supports multicast communications. We show how QUIC features and built-in security can be leveraged for multicast transport. We present the design of MCQUIC and implement it in Cloudflare quiche. We assess its performance through benchmarks and in emulated networks under realistic scenarios. We also demonstrate MCQUIC in a campus network. By coupling QUIC with our multicast extension, applications can rely on multicast for efficiency with the possibility to fall back on unicast in case of incompatible network conditions.
Exp[licit]-A Robot modeling Software based on Exponential Maps
Venue: arXiv cs.RO 2023
Year: 2023

$ $Deriving a robot's equation of motion typically requires placing multiple coordinate frames, commonly using the Denavit-Hartenberg convention to express the kinematic and dynamic relationships between segments. This paper presents an alternative using the differential geometric method of Exponential Maps, which reduces the number of coordinate frame choices to two. The traditional and differential geometric methods are compared, and the conceptual and practical differences are detailed. The open-source software, Exp[licit], based on the differential geometric method, is introduced. It is intended for use by researchers and engineers with basic knowledge of geometry and robotics. Code snippets and an example application are provided to demonstrate the benefits of the differential geometric method and assist users to get started with the software.
Beyond English: Centering Multilingualism in Data Visualization
Venue: arXiv cs.HC 2023
Year: 2023

Information visualization and natural language are intricately linked. However, the majority of research and relevant work in information and data visualization (and human-computer interaction) involve English-speaking populations as both researchers and participants, are published in English, and are presented predominantly at English-speaking venues. Although several solutions can be proposed such as translating English texts in visualization to other languages, there is little research that looks at the intersection of data visualization and different languages, and the implications that current visualization practices have on non-English speaking communities. In this position paper, we argue that linguistically diverse communities abound beyond the English-speaking world and offer a richness of experiences for the visualization research community to engage with. Through a case study of how two non-English languages interplay with data visualization reasoning in Madagascar, we describe how monolingualism in data visualization impacts the experiences of underrepresented populations and emphasize potential harm to these communities. Lastly, we raise several questions towards advocating for more inclusive visualization practices that center the diverse experiences of linguistically underrepresented populations.
SHARM: Segmented Head Anatomical Reference Models
Venue: arXiv cs.CV 2023
Year: 2023

Reliable segmentation of anatomical tissues of human head is a major step in several clinical applications such as brain mapping, surgery planning and associated computational simulation studies. Segmentation is based on identifying different anatomical structures through labeling different tissues through medical imaging modalities. The segmentation of brain structures is commonly feasible with several remarkable contributions mainly for medical perspective; however, non-brain tissues are of less interest due to anatomical complexity and difficulties to be observed using standard medical imaging protocols. The lack of whole head segmentation methods and unavailability of large human head segmented datasets limiting the variability studies, especially in the computational evaluation of electrical brain stimulation (neuromodulation), human protection from electromagnetic field, and electroencephalography where non-brain tissues are of great importance. To fill this gap, this study provides an open-access Segmented Head Anatomical Reference Models (SHARM) that consists of 196 subjects. These models are segmented into 15 different tissues; skin, fat, muscle, skull cancellous bone, skull cortical bone, brain white matter, brain gray matter, cerebellum white matter, cerebellum gray matter, cerebrospinal fluid, dura, vitreous humor, lens, mucous tissue and blood vessels. The segmented head models are generated using open-access IXI MRI dataset through convolutional neural network structure named ForkNet+. Results indicate a high consistency in statistical characteristics of different tissue distribution in age scale with real measurements. SHARM is expected to be a useful benchmark not only for electromagnetic dosimetry studies but also for different human head segmentation applications.
A Novel Low-Cost, Recyclable, Easy-to-Build Robot Blimp For Transporting Supplies in Hard-to-Reach Locations
Venue: arXiv cs.RO 2023
Year: 2023

Rural communities in remote areas often encounter significant challenges when it comes to accessing emergency healthcare services and essential supplies due to a lack of adequate transportation infrastructure. The situation is further exacerbated by poorly maintained, damaged, or flooded roads, making it arduous for rural residents to obtain the necessary aid in critical situations. Limited budgets and technological constraints pose additional obstacles, hindering the prompt response of local rescue teams during emergencies. The transportation of crucial resources, such as medical supplies and food, plays a vital role in saving lives in these situations. In light of these obstacles, our objective is to improve accessibility and alleviate the suffering of vulnerable populations by automating transportation tasks using low-cost robotic systems. We propose a low-cost, easy-to-build blimp robot (UAVs), that can significantly enhance the efficiency and effectiveness of local emergency responses.
Designing Voice Interfaces to Support Mindfulness-Based Pain Management
Venue: arXiv cs.HC 2023
Year: 2023

Objective: Chronic pain is a critical public health issue affecting approximately 20% of the adult population in the United States. Given the opioid crisis, there has been an urgent focus on non-addictive pain management methods including Mindfulness-Based Stress Reduction (MBSR). Prior work has successfully used MBSR for pain management. However, ensuring longitudinal engagement to MBSR practices remains a serious challenge. In this work, we explore the utility of a voice interface to support MBSR home practice. Methods: We interviewed ten mindfulness program facilitators to understand how such a technology might fit in the context of the MBSR class and identify potential usability issues with our prototype. We then used directed content analysis to identify key themes and sub-themes within the interview data. Results: Our findings show that facilitators supported the use of the voice interface for MBSR, particularly for individuals with limited motor function. Facilitators also highlighted unique affordances of voice interfaces, including perceived social presence, to support sustained engagement. Conclusion: We demonstrate the acceptability of a voice interface to support home practice for MBSR participants among trained mindfulness facilitators. Based on our findings, we outline design recommendations for technologies aiming to provide longitudinal support for mindfulness-based interventions. Future work should further these efforts towards making non-addictive pain management interventions accessible and efficacious for a wide audience of users.
Deep Attentive Time Warping
Venue: arXiv cs.CV 2023
Year: 2023

Similarity measures for time series are important problems for time series classification. To handle the nonlinear time distortions, Dynamic Time Warping (DTW) has been widely used. However, DTW is not learnable and suffers from a trade-off between robustness against time distortion and discriminative power. In this paper, we propose a neural network model for task-adaptive time warping. Specifically, we use the attention model, called the bipartite attention model, to develop an explicit time warping mechanism with greater distortion invariance. Unlike other learnable models using DTW for warping, our model predicts all local correspondences between two time series and is trained based on metric learning, which enables it to learn the optimal data-dependent warping for the target task. We also propose to induce pre-training of our model by DTW to improve the discriminative power. Extensive experiments demonstrate the superior effectiveness of our model over DTW and its state-of-the-art performance in online signature verification.
Dynamic Spectrum Mixer for Visual Recognition
Venue: arXiv cs.CV 2023
Year: 2023

Recently, MLP-based vision backbones have achieved promising performance in several visual recognition tasks. However, the existing MLP-based methods directly aggregate tokens with static weights, leaving the adaptability to different images untouched. Moreover, Recent research demonstrates that MLP-Transformer is great at creating long-range dependencies but ineffective at catching high frequencies that primarily transmit local information, which prevents it from applying to the downstream dense prediction tasks, such as semantic segmentation. To address these challenges, we propose a content-adaptive yet computationally efficient structure, dubbed Dynamic Spectrum Mixer (DSM). The DSM represents token interactions in the frequency domain by employing the Discrete Cosine Transform, which can learn long-term spatial dependencies with log-linear complexity. Furthermore, a dynamic spectrum weight generation layer is proposed as the spectrum bands selector, which could emphasize the informative frequency bands while diminishing others. To this end, the technique can efficiently learn detailed features from visual input that contains both high- and low-frequency information. Extensive experiments show that DSM is a powerful and adaptable backbone for a range of visual recognition tasks. Particularly, DSM outperforms previous transformer-based and MLP-based models, on image classification, object detection, and semantic segmentation tasks, such as 83.8 \% top-1 accuracy on ImageNet, and 49.9 \% mIoU on ADE20K.
Deep Nonparametric Convexified Filtering for Computational Photography, Image Synthesis and Adversarial Defense
Venue: arXiv cs.CV 2023
Year: 2023

We aim to provide a general framework of for computational photography that recovers the real scene from imperfect images, via the Deep Nonparametric Convexified Filtering (DNCF). It is consists of a nonparametric deep network to resemble the physical equations behind the image formation, such as denoising, super-resolution, inpainting, and flash. DNCF has no parameterization dependent on training data, therefore has a strong generalization and robustness to adversarial image manipulation. During inference, we also encourage the network parameters to be nonnegative and create a bi-convex function on the input and parameters, and this adapts to second-order optimization algorithms with insufficient running time, having 10X acceleration over Deep Image Prior. With these tools, we empirically verify its capability to defend image classification deep networks against adversary attack algorithms in real-time.
Remote Sensing Object Detection Meets Deep Learning: A Meta-review of Challenges and Advances
Venue: arXiv cs.CV 2023
Year: 2023

Remote sensing object detection (RSOD), one of the most fundamental and challenging tasks in the remote sensing field, has received longstanding attention. In recent years, deep learning techniques have demonstrated robust feature representation capabilities and led to a big leap in the development of RSOD techniques. In this era of rapid technical evolution, this review aims to present a comprehensive review of the recent achievements in deep learning based RSOD methods. More than 300 papers are covered in this review. We identify five main challenges in RSOD, including multi-scale object detection, rotated object detection, weak object detection, tiny object detection, and object detection with limited supervision, and systematically review the corresponding methods developed in a hierarchical division manner. We also review the widely used benchmark datasets and evaluation metrics within the field of RSOD, as well as the application scenarios for RSOD. Future research directions are provided for further promoting the research in RSOD.
Hierarchical Time-Optimal Planning for Multi-Vehicle Racing
Venue: arXiv cs.RO 2023
Year: 2023

This paper presents a hierarchical planning algorithm for racing with multiple opponents. The two-stage approach consists of a high-level behavioral planning step and a low-level optimization step. By combining discrete and continuous planning methods, our algorithm encourages global time optimality without being limited by coarse discretization. In the behavioral planning step, the fastest behavior is determined with a low-resolution spatio-temporal visibility graph. Based on the selected behavior, we calculate maneuver envelopes that are subsequently applied as constraints in a time-optimal control problem. The performance of our method is comparable to a parallel approach that selects the fastest trajectory from multiple optimizations with different behavior classes. However, our algorithm can be executed on a single core. This significantly reduces computational requirements, especially when multiple opponents are involved. Therefore, the proposed method is an efficient and practical solution for real-time multi-vehicle racing scenarios.
DCTTS: Discrete Diffusion Model with Contrastive Learning for Text-to-speech Generation
Venue: arXiv cs.SD 2023
Year: 2023

In the Text-to-speech(TTS) task, the latent diffusion model has excellent fidelity and generalization, but its expensive resource consumption and slow inference speed have always been a challenging. This paper proposes Discrete Diffusion Model with Contrastive Learning for Text-to-Speech Generation(DCTTS). The following contributions are made by DCTTS: 1) The TTS diffusion model based on discrete space significantly lowers the computational consumption of the diffusion model and improves sampling speed; 2) The contrastive learning method based on discrete space is used to enhance the alignment connection between speech and text and improve sampling quality; and 3) It uses an efficient text encoder to simplify the model's parameters and increase computational efficiency. The experimental results demonstrate that the approach proposed in this paper has outstanding speech synthesis quality and sampling speed while significantly reducing the resource consumption of diffusion model. The synthesized samples are available at https://github.com/lawtherWu/DCTTS.
TAP: Targeted Prompting for Task Adaptive Generation of Textual Training Instances for Visual Classification
Venue: arXiv cs.CV 2023
Year: 2023

Vision and Language Models (VLMs), such as CLIP, have enabled visual recognition of a potentially unlimited set of categories described by text prompts. However, for the best visual recognition performance, these models still require tuning to better fit the data distributions of the downstream tasks, in order to overcome the domain shift from the web-based pre-training data. Recently, it has been shown that it is possible to effectively tune VLMs without any paired data, and in particular to effectively improve VLMs visual recognition performance using text-only training data generated by Large Language Models (LLMs). In this paper, we dive deeper into this exciting text-only VLM training approach and explore ways it can be significantly further improved taking the specifics of the downstream task into account when sampling text data from LLMs. In particular, compared to the SOTA text-only VLM training approach, we demonstrate up to 8.4% performance improvement in (cross) domain-specific adaptation, up to 8.7% improvement in fine-grained recognition, and 3.1% overall average improvement in zero-shot classification compared to strong baselines.
Information design in Bayesian routing games
Venue: arXiv cs.GT 2023
Year: 2023

We study optimal information provision in transportation networks when users are strategic and the network state is uncertain. An omniscient planner observes the network state and discloses information to the users with the goal of minimizing the expected travel time at the user equilibrium. Public signal policies, including full-information disclosure, are known to be inefficient in achieving optimality. For this reason, we focus on private signals and restrict without loss of generality the analysis to signals that coincide with path recommendations that satisfy obedience constraints, namely users have no incentive in deviating from the received recommendation according to their posterior belief. We first formulate the general problem and analyze its properties for arbitrary network topologies and delay functions. Then, we consider the case of two parallel links with affine delay functions, and provide sufficient conditions under which optimality can be achieved by information design. Interestingly, we observe that the system benefits from uncertainty, namely it is easier for the planner to achieve optimality when the variance of the uncertain parameters is large. We then provide an example where optimality can be achieved even if the sufficient conditions for optimality are not met.
OWL Reasoners still useable in 2023
Venue: arXiv cs.AI 2023
Year: 2023

In a systematic literature and software review over 100 OWL reasoners/systems were analyzed to see if they would still be usable in 2023. This has never been done in this capacity. OWL reasoners still play an important role in knowledge organisation and management, but the last comprehensive surveys/studies are more than 8 years old. The result of this work is a comprehensive list of 95 standalone OWL reasoners and systems using an OWL reasoner. For each item, information on project pages, source code repositories and related documentation was gathered. The raw research data is provided in a Github repository for anyone to use.
DreamStyler: Paint by Style Inversion with Text-to-Image Diffusion Models
Venue: arXiv cs.CV 2023
Year: 2023

Recent progresses in large-scale text-to-image models have yielded remarkable accomplishments, finding various applications in art domain. However, expressing unique characteristics of an artwork (e.g. brushwork, colortone, or composition) with text prompts alone may encounter limitations due to the inherent constraints of verbal description. To this end, we introduce DreamStyler, a novel framework designed for artistic image synthesis, proficient in both text-to-image synthesis and style transfer. DreamStyler optimizes a multi-stage textual embedding with a context-aware text prompt, resulting in prominent image quality. In addition, with content and style guidance, DreamStyler exhibits flexibility to accommodate a range of style references. Experimental results demonstrate its superior performance across multiple scenarios, suggesting its promising potential in artistic product creation.
Enhancing the Performance of Multi-Agent Reinforcement Learning for Controlling HVAC Systems
Venue: arXiv cs.MA 2023
Year: 2023

Systems for heating, ventilation and air-conditioning (HVAC) of buildings are traditionally controlled by a rule-based approach. In order to reduce the energy consumption and the environmental impact of HVAC systems more advanced control methods such as reinforcement learning are promising. Reinforcement learning (RL) strategies offer a good alternative, as user feedback can be integrated more easily and presence can also be incorporated. Moreover, multi-agent RL approaches scale well and can be generalized. In this paper, we propose a multi-agent RL framework based on existing work that learns reducing on one hand energy consumption by optimizing HVAC control and on the other hand user feedback by occupants about uncomfortable room temperatures. Second, we show how to reduce training time required for proper RL-agent-training by using parameter sharing between the multiple agents and apply different pretraining techniques. Results show that our framework is capable of reducing the energy by around 6% when controlling a complete building or 8% for a single room zone. The occupants complaints are acceptable or even better compared to a rule-based baseline. Additionally, our performance analysis show that the training time can be drastically reduced by using parameter sharing.