pwc-archive/papers-with-abstracts huggingface dataset:
Distributed multi-robot collision avoidance via deep reinforcement learning for navigation in complex scenarios
Authors: Tingxiang Fan, Pinxin Long, Wenxi Liu and Jia Pan
Date: 2020-05-31 00:00:00
Proceeding: The International Journal of Robotics Research 2020 5
Tasks: Autonomous Navigation, Collision Avoidance, Deep Reinforcement Learning, Robot Navigation

Developing a safe and efficient collision-avoidance policy for multiple robots is challenging in the decentralized scenarios
where each robot generates its paths with limited observation of other robots’ states and intentions. Prior distributed
multi-robot collision-avoidance systems often require frequent inter-robot communication or agent-level features to plan a
local collision-free action, which is not robust and computationally prohibitive. In addition, the performance of these
methods is not comparable with their centralized counterparts in practice. In this article, we present a decentralized
sensor-level collision-avoidance policy for multi-robot systems, which shows promising results in practical applications.
In particular, our policy directly maps raw sensor measurements to an agent’s steering commands in terms of the movement
velocity. As a first step toward reducing the performance gap between decentralized and centralized methods, we
present a multi-scenario multi-stage training framework to learn an optimal policy. The policy is trained over a large
number of robots in rich, complex environments simultaneously using a policy-gradient-based reinforcement-learning
algorithm. The learning algorithm is also integrated into a hybrid control framework to further improve the policy’s
robustness and effectiveness. We validate the learned sensor-level collision-3avoidance policy in a variety of simulated
and real-world scenarios with thorough performance evaluations for large-scale multi-robot systems. The generalization
of the learned policy is verified in a set of unseen scenarios including the navigation of a group of heterogeneous robots
and a large-scale scenario with 100 robots. Although the policy is trained using simulation data only, we have successfully
deployed it on physical robots with shapes and dynamics characteristics that are different from the simulated agents,
in order to demonstrate the controller’s robustness against the simulation-to-real modeling error. Finally, we show that
the collision-avoidance policy learned from multi-robot navigation tasks provides an excellent solution for safe and effective
autonomous navigation for a single robot working in a dense real human crowd. Our learned policy enables a robot
to make effective progress in a crowd without getting stuck. More importantly, the policy has been successfully deployed
on different types of physical robot platforms without tedious parameter tuning. Videos are available at https://sites.google.
com/view/hybridmrca.
Deep Attention Network for Enhanced Hand Gesture Recognition System
Authors: Priscilla Dinkar Moyya1  · Kiran Kumar Neelamsetti2  · Gopalakrishnan Thirumoorthy3
Date: 2023-04-15 00:00:00
Proceeding: journal 2023 4
Tasks: Deep Attention, Gesture Recognition, Hand Gesture Recognition, Hand-Gesture Recognition

Since technology is growing in all felds, researchers are developing more advanced technologies in human–computer communication and security systems. The fexible wrist hinge and crowded background make it difcult to distinguish hands 
in uncontrolled conditions. We proposed a deep attention network, that can automatically localize the hand and classify 
the gestures. Available datasets were collected to train the deep neural network architecture to create a gesture recognition 
system. The datasets used are 20-bn-jester and NTU-HD datasets. 20-bn-jester dataset is RGB dataset and NTU-HD dataset 
is RGB-D dataset. The features are extracted from the dataset and fed into the models for training and classifcation. Two 
models Resnet-50 and VGG-16 are used for training and classifcation. In this work, a special network used for localizing 
hand regions is a deep attention network. Using an attention network, gesture recognition system has been created for static 
RGB-D images. The proposed architecture yielded a better accuracy of 91.32% using Resnet-50, and 89% using VGG-16, 
respectively. The signifcance of these results shows that one can create a hand gesture recognition system feasibly and 
prominently.
EDOLAB: An Open-Source Platform for Education and Experimentation with Evolutionary Dynamic Optimization Algorithms
Authors: Mai Peng, Delaram Yazdani, Zeneng She, Danial Yazdani, Wenjian Luo, Changhe Li, Juergen Branke, Trung Thanh Nguyen, Amir H. Gandomi, Shengxiang Yang, Yaochu Jin, Xin Yao
Date: 2023-08-24 00:00:00

Many real-world optimization problems exhibit dynamic characteristics, posing significant challenges for traditional optimization techniques. Evolutionary Dynamic Optimization Algorithms (EDOAs) are designed to address these challenges effectively. However, in existing literature, the reported results for a given EDOA can vary significantly. This inconsistency often arises because the source codes for many EDOAs, which are typically complex, have not been made publicly available, leading to error-prone re-implementations. To support researchers in conducting experiments and comparing their algorithms with various EDOAs, we have developed an open-source MATLAB platform called the Evolutionary Dynamic Optimization LABoratory (EDOLAB). This platform not only facilitates research but also includes an educational module designed for instructional purposes. The education module allows users to observe: a) a 2-dimensional problem space and its morphological changes following each environmental change, b) the behaviors of individuals over time, and c) how the EDOA responds to environmental changes and tracks the moving optimum. The current version of EDOLAB features 25 EDOAs and four fully parametric benchmark generators. The MATLAB source code for EDOLAB is publicly available and can be accessed from [https://github.com/Danial-Yazdani/EDOLAB-MATLAB].
HuBo-VLM: Unified Vision-Language Model designed for HUman roBOt interaction tasks
Authors: ZiChao Dong, Weikun Zhang, Xufeng Huang, Hang Ji, Xin Zhan, Junbo Chen
Date: 2023-08-24 00:00:00
Tasks: Language Modeling, Language Modelling, LIDAR Semantic Segmentation, object-detection, Object Detection, Referring Expression Comprehension, Visual Grounding

Human robot interaction is an exciting task, which aimed to guide robots following instructions from human. Since huge gap lies between human natural language and machine codes, end to end human robot interaction models is fair challenging. Further, visual information receiving from sensors of robot is also a hard language for robot to perceive. In this work, HuBo-VLM is proposed to tackle perception tasks associated with human robot interaction including object detection and visual grounding by a unified transformer based vision language model. Extensive experiments on the Talk2Car benchmark demonstrate the effectiveness of our approach. Code would be publicly available in https://github.com/dzcgaara/HuBo-VLM.
VIGC: Visual Instruction Generation and Correction
Authors: Bin Wang, Fan Wu, Xiao Han, Jiahui Peng, Huaping Zhong, Pan Zhang, Xiaoyi Dong, Weijia Li, Wei Li, Jiaqi Wang, Conghui He
Date: 2023-08-24 00:00:00
Tasks: Hallucination, Image Captioning, Language Modelling

The integration of visual encoders and large language models (LLMs) has driven recent progress in multimodal large language models (MLLMs). However, the scarcity of high-quality instruction-tuning data for vision-language tasks remains a challenge. The current leading paradigm, such as LLaVA, relies on language-only GPT-4 to generate data, which requires pre-annotated image captions and detection bounding boxes, suffering from understanding image details. A practical solution to this problem would be to utilize the available multimodal large language models (MLLMs) to generate instruction data for vision-language tasks. However, it's worth noting that the currently accessible MLLMs are not as powerful as their LLM counterparts, as they tend to produce inadequate responses and generate false information. As a solution for addressing the current issue, this paper proposes the Visual Instruction Generation and Correction (VIGC) framework that enables multimodal large language models to generate instruction-tuning data and progressively enhance its quality on-the-fly. Specifically, Visual Instruction Generation (VIG) guides the vision-language model to generate diverse instruction-tuning data. To ensure generation quality, Visual Instruction Correction (VIC) adopts an iterative update mechanism to correct any inaccuracies in data produced by VIG, effectively reducing the risk of hallucination. Leveraging the diverse, high-quality data generated by VIGC, we finetune mainstream models and validate data quality based on various evaluations. Experimental results demonstrate that VIGC not only compensates for the shortcomings of language-only data generation methods, but also effectively enhances the benchmark performance. The models, datasets, and code are available at https://opendatalab.github.io/VIGC.
Deep learning for non-parameterized MEMS structural design
Authors: Ruiqi Guo, Fanping Sui, Wei Yue, Zekai Wang, Sedat Pala, Kunying Li, Renxiao Xu, Liwei Lin
Date: 2022-08-29 00:00:00
Proceeding: Nature 2022 8
Tasks: Deep Learning

The geometric designs of MEMS devices can profoundly impact their physical properties and eventual performances.
However, it is challenging for researchers to rationally consider a large number of possible designs, as it would be very
time- and resource-consuming to study all these cases using numerical simulation. In this paper, we report the use of
deep learning techniques to accelerate the MEMS design cycle by quickly and accurately predicting the physical
properties of numerous design candidates with vastly different geometric features. Design candidates are represented
in a nonparameterized, topologically unconstrained form using pixelated black-and-white images. After sufficient
training, a deep neural network can quickly calculate the physical properties of interest with good accuracy without
using conventional numerical tools such as finite element analysis. As an example, we apply our deep learning
approach in the prediction of the modal frequency and quality factor of disk-shaped microscale resonators. With
reasonable training, our deep learning neural network becomes a high-speed, high-accuracy calculator: it can identify
the flexural mode frequency and the quality factor 4.6 × 10 times and 2.6 × 10 times faster, respectively, than
conventional numerical simulation packages, with good accuracies of 98.8 ± 1.6% and 96.8 ± 3.1%, respectively. When
simultaneously predicting the frequency and the quality factor, up to ~96.0% of the total computation time can be
saved during the design process. The proposed technique can rapidly screen over thousands of design candidates and
promotes experience-free and data-driven MEMS structural designs.
NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenes
Authors: Muhammad Zubair Irshad, Sergey Zakharov, Katherine Liu, Vitor Guizilini, Thomas Kollar, Adrien Gaidon, Zsolt Kira, Rares Ambrus
Date: 2023-08-24 00:00:00
Proceeding: ICCV 2023 1
Tasks: Generalizable Novel View Synthesis, Novel View Synthesis

Recent implicit neural representations have shown great results for novel view synthesis. However, existing methods require expensive per-scene optimization from many views hence limiting their application to real-world unbounded urban settings where the objects of interest or backgrounds are observed from very few views. To mitigate this challenge, we introduce a new approach called NeO 360, Neural fields for sparse view synthesis of outdoor scenes. NeO 360 is a generalizable method that reconstructs 360{\deg} scenes from a single or a few posed RGB images. The essence of our approach is in capturing the distribution of complex real-world outdoor 3D scenes and using a hybrid image-conditional triplanar representation that can be queried from any world point. Our representation combines the best of both voxel-based and bird's-eye-view (BEV) representations and is more effective and expressive than each. NeO 360's representation allows us to learn from a large collection of unbounded 3D scenes while offering generalizability to new views and novel scenes from as few as a single image during inference. We demonstrate our approach on the proposed challenging 360{\deg} unbounded dataset, called NeRDS 360, and show that NeO 360 outperforms state-of-the-art generalizable methods for novel view synthesis while also offering editing and composition capabilities. Project page: https://zubair-irshad.github.io/projects/neo360.html
DLIP: Distilling Language-Image Pre-training
Authors: Huafeng Kuang, Jie Wu, Xiawu Zheng, Ming Li, Xuefeng Xiao, Rui Wang, Min Zheng, Rongrong Ji
Date: 2023-08-24 00:00:00
Tasks: Image Captioning, Image-text Retrieval, Knowledge Distillation, Model Compression, Question Answering, Retrieval, Text Retrieval, Visual Question Answering

Vision-Language Pre-training (VLP) shows remarkable progress with the assistance of extremely heavy parameters, which challenges deployment in real applications. Knowledge distillation is well recognized as the essential procedure in model compression. However, existing knowledge distillation techniques lack an in-depth investigation and analysis of VLP, and practical guidelines for VLP-oriented distillation are still not yet explored. In this paper, we present DLIP, a simple yet efficient Distilling Language-Image Pre-training framework, through which we investigate how to distill a light VLP model. Specifically, we dissect the model distillation from multiple dimensions, such as the architecture characteristics of different modules and the information transfer of different modalities. We conduct comprehensive experiments and provide insights on distilling a light but performant VLP model. Experimental results reveal that DLIP can achieve a state-of-the-art accuracy/efficiency trade-off across diverse cross-modal tasks, e.g., image-text retrieval, image captioning and visual question answering. For example, DLIP compresses BLIP by 1.9x, from 213M to 108M parameters, while achieving comparable or better performance. Furthermore, DLIP succeeds in retaining more than 95% of the performance with 22.4% parameters and 24.8% FLOPs compared to the teacher model and accelerates inference speed by 2.7x.
Low-count Time Series Anomaly Detection
Authors: Philipp Renz, Kurt Cutajar, Niall Twomey, Gavin K. C. Cheung, Hanting Xie
Date: 2023-08-24 00:00:00
Tasks: Anomaly Detection, Time Series, Time Series Anomaly Detection

Low-count time series describe sparse or intermittent events, which are prevalent in large-scale online platforms that capture and monitor diverse data types. Several distinct challenges surface when modelling low-count time series, particularly low signal-to-noise ratios (when anomaly signatures are provably undetectable), and non-uniform performance (when average metrics are not representative of local behaviour). The time series anomaly detection community currently lacks explicit tooling and processes to model and reliably detect anomalies in these settings. We address this gap by introducing a novel generative procedure for creating benchmark datasets comprising of low-count time series with anomalous segments. Via a mixture of theoretical and empirical analysis, our work explains how widely-used algorithms struggle with the distribution overlap between normal and anomalous segments. In order to mitigate this shortcoming, we then leverage our findings to demonstrate how anomaly score smoothing consistently improves performance. The practical utility of our analysis and recommendation is validated on a real-world dataset containing sales data for retail stores.
Evaluating the Vulnerabilities in ML systems in terms of adversarial attacks
Authors: John Harshith, Mantej Singh Gill, Madhan Jothimani
Date: 2023-08-24 00:00:00

There have been recent adversarial attacks that are difficult to find. These new adversarial attacks methods may pose challenges to current deep learning cyber defense systems and could influence the future defense of cyberattacks. The authors focus on this domain in this research paper. They explore the consequences of vulnerabilities in AI systems. This includes discussing how they might arise, differences between randomized and adversarial examples and also potential ethical implications of vulnerabilities. Moreover, it is important to train the AI systems appropriately when they are in testing phase and getting them ready for broader use.
Language as Reality: A Co-Creative Storytelling Game Experience in 1001 Nights using Generative AI
Authors: Yuqian Sun, Zhouyi Li, Ke Fang, Chang Hee Lee, Ali Asadipour
Date: 2023-08-24 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model

In this paper, we present "1001 Nights", an AI-native game that allows players lead in-game reality through co-created storytelling with the character driven by large language model. The concept is inspired by Wittgenstein's idea of the limits of one's world being determined by the bounds of their language. Using advanced AI tools like GPT-4 and Stable Diffusion, the second iteration of the game enables the protagonist, Shahrzad, to realize words and stories in her world. The player can steer the conversation with the AI King towards specific keywords, which then become battle equipment in the game. This blend of interactive narrative and text-to-image transformation challenges the conventional border between the game world and reality through a dual perspective. We focus on Shahrzad, who seeks to alter her fate compared to the original folklore, and the player, who collaborates with AI to craft narratives and shape the game world. We explore the technical and design elements of implementing such a game with an objective to enhance the narrative game genre with AI-generated content and to delve into AI-native gameplay possibilities.
CDAN: Convolutional dense attention-guided network for low-light image enhancement
Authors: Hossein Shakibania, Sina Raoufi, Hassan Khotanlou
Date: 2023-08-24 00:00:00
Tasks: Image Enhancement, Low-Light Image Enhancement

Low-light images, characterized by inadequate illumination, pose challenges of diminished clarity, muted colors, and reduced details. Low-light image enhancement, an essential task in computer vision, aims to rectify these issues by improving brightness, contrast, and overall perceptual quality, thereby facilitating accurate analysis and interpretation. This paper introduces the Convolutional Dense Attention-guided Network (CDAN), a novel solution for enhancing low-light images. CDAN integrates an autoencoder-based architecture with convolutional and dense blocks, complemented by an attention mechanism and skip connections. This architecture ensures efficient information propagation and feature learning. Furthermore, a dedicated post-processing phase refines color balance and contrast. Our approach demonstrates notable progress compared to state-of-the-art results in low-light image enhancement, showcasing its robustness across a wide range of challenging scenarios. Our model performs remarkably on benchmark datasets, effectively mitigating under-exposure and proficiently restoring textures and colors in diverse low-light scenarios. This achievement underscores CDAN's potential for diverse computer vision tasks, notably enabling robust object detection and recognition in challenging low-light conditions.
Can Linguistic Knowledge Improve Multimodal Alignment in Vision-Language Pretraining?
Authors: Fei Wang, Liang Ding, Jun Rao, Ye Liu, Li Shen, Changxing Ding
Date: 2023-08-24 00:00:00
Tasks: Attribute, Negation, Sentence

The multimedia community has shown a significant interest in perceiving and representing the physical world with multimodal pretrained neural network models, and among them, the visual-language pertaining (VLP) is, currently, the most captivating topic. However, there have been few endeavors dedicated to the exploration of 1) whether essential linguistic knowledge (e.g., semantics and syntax) can be extracted during VLP, and 2) how such linguistic knowledge impact or enhance the multimodal alignment. In response, here we aim to elucidate the impact of comprehensive linguistic knowledge, including semantic expression and syntactic structure, on multimodal alignment. Specifically, we design and release the SNARE, the first large-scale multimodal alignment probing benchmark, to detect the vital linguistic components, e.g., lexical, semantic, and syntax knowledge, containing four tasks: Semantic structure, Negation logic, Attribute ownership, and Relationship composition. Based on our proposed probing benchmarks, our holistic analyses of five advanced VLP models illustrate that the VLP model: i) shows insensitivity towards complex syntax structures and relies on content words for sentence comprehension; ii) demonstrates limited comprehension of combinations between sentences and negations; iii) faces challenges in determining the presence of actions or spatial relationships within visual information and struggles with verifying the correctness of triple combinations. We make our benchmark and code available at \url{https://github.com/WangFei-2019/SNARE/}.
Large Language Models Vote: Prompting for Rare Disease Identification
Authors: David Oniani, Jordan Hilsman, Hang Dong, Fengyi Gao, Shiven Verma, Yanshan Wang
Date: 2023-08-24 00:00:00
Tasks: Few-Shot Learning

The emergence of generative Large Language Models (LLMs) emphasizes the need for accurate and efficient prompting approaches. LLMs are often applied in Few-Shot Learning (FSL) contexts, where tasks are executed with minimal training data. FSL has become popular in many Artificial Intelligence (AI) subdomains, including AI for health. Rare diseases affect a small fraction of the population. Rare disease identification from clinical notes inherently requires FSL techniques due to limited data availability. Manual data collection and annotation is both expensive and time-consuming. In this paper, we propose Models-Vote Prompting (MVP), a flexible prompting approach for improving the performance of LLM queries in FSL settings. MVP works by prompting numerous LLMs to perform the same tasks and then conducting a majority vote on the resulting outputs. This method achieves improved results to any one model in the ensemble on one-shot rare disease identification and classification tasks. We also release a novel rare disease dataset for FSL, available to those who signed the MIMIC-IV Data Use Agreement (DUA). Furthermore, in using MVP, each model is prompted multiple times, substantially increasing the time needed for manual annotation, and to address this, we assess the feasibility of using JSON for automating generative LLM evaluation.