Reviewerly/exHarmony huggingface dataset:
Multi-Target Encirclement with Collision Avoidance via Deep Reinforcement Learning using Relational Graphs

In this paper, we propose a novel decentralized method based on deep reinforcement learning using robot-level and target-level relational graphs, to solve the problem of multi-target encirclement with collision avoidance (MECA). Specifically, the robot-level relational graphs, composed of three heterogeneous relational graphs between each robot and other robots, targets and obstacles, are modeled and learned through using graph attention networks (GATs) for extracting different spatial relational representations. Moreover, for each target within the observation of each robot, a target-level relational graph is built with GAT to construct spatial relations from the robot. Furthermore, the movement of each target is modeled by the target-level relational graph and learned through supervised learning for predicting the trajectory of the target. In addition, a knowledge-embedded compound reward function is defined to solve the multi-objective problem in MECA, and guide the policy learning for deriving the behavior of MECA. An actor-critic training algorithm based on the centralized training and decentralized execution framework is adopted to train the policy network. Simulation and real-world experiment results demonstrate the effectiveness and generalization of our method.
An Assembly Sequence Planning Framework for Complex Data using General Voronoi Diagram

We present the first realization of an assembly sequence planning framework for large-scale and complex 3D real-world CAD scenarios. Other than in academic benchmark data sets, in our scenario each assembled part is allowed to contain flexible fastening elements and the number of assembled parts is quite high. With our framework we are able to derive a meaningful assembly priority graph for the parts. Our framework divides the disassembly motion of each part into a NEAR- and a subsequent FAR planning phase and uses existing specialized motion planners for each phase. To reduce the number of unsuccessful motion planning requests we use a general Voronoi diagram graph and a novel collision perceiving method which significantly speed up our framework. At the end, we create an assembly priority graph to indicate which parts must be disassembled before others. In our experiments, we show that our framework is the first one which is able to generate a priority graph for a representative data set from the automotive industry. Moreover, the reported disassembly motions for the individual parts are shorter and can be computed faster than with other state-of-the-art frameworks.
Automated Linear and Non-Linear Path Planning for Neurosurgical Interventions

Recent advances in medical technology have produced a number of flexible instruments that are capable of traversing non-linear paths. This is of special interest in the field of neurosurgery. However, the non-rigid instruments have the disadvantage that path planning becomes increasingly difficult. In addition to anatomical risk factors, the mechanical properties and constraints of the specific instrument must also be considered. To support surgeons to deal with the increase in planning complexity, we present a novel method for both linear and arbitrary follow-the-leader flexible path planning. Our method is utilizing patient-specific image data, which is then used to generate a multi-objective problem consisting of conventional risk metrics for path planning in high-risk regions like the accumulated path cost or the distance to risk structures. Simultaneously, the path-problem is also constraint to mechanical properties of the instrument such as curvature or maximum operational length. Optimal paths can then be generated by solving a multi-objective problem by approximating the Pareto front. We show that our method can automatically generate linear and non-linear paths for neurosurgical interventions in the human brain in less than 2 minutes. Furthermore, we show that the proposed automated method generates paths with 87% reduced risk compared to standard of care plannings.
FusionNet: Coarse-to-Fine Extrinsic Calibration Network of LiDAR and Camera with Hierarchical Point-pixel Fusion

In this paper, we propose a novel network, Fusion-Net, which can estimate the extrinsic calibration matrix between LiDAR and a monocular RGB camera with high accuracy and robustness. FusionNet is a coarse-to-fine method, providing an online and end-to-end solution that can automatically detect and correct the decalibration without any specially designed targets or environments. First, the network applies deep-learning-based technologies to extract the features of LiDAR point clouds and RGB images. Then a novel method is adopted to fuse the features got from different sensors by projecting LiDAR features onto RGB feature maps, searching for the RGB features with the projected points as centers and concatenating the extracted RGB features with LiDAR features. To increase the accuracy, we apply a coarse-to-fine method in the network, by transforming LiDAR points and estimating the extrinsic calibration matrices from the coarse scale to the fine scale. The network is trained on random artificial decalibration matrices. Compared to existing approaches, our method doesn't need to train additional iterative networks, but it can also adapt to different ranges of decalibration.
Safe multi-agent motion planning via filtered reinforcement learning

We study the problem of safe multi-agent motion planning in cluttered environments. Existing multi-agent reinforcement learning-based motion planners only provide approximate safety enforcement. We propose a safe reinforcement learning algorithm that leverages single-agent reinforcement learning for target regulation and a subsequent convex optimization-based filtering that ensures the collective safety of the system. Our approach yields a safe, real-time implementable multi-agent motion planner that is simpler to train and enforces safety as hard constraints. Our approach can handle state and control constraints on the agents, and enforce collision avoidance among themselves and with static obstacles in the environment. Numerical simulations and hardware experiments show the efficacy of the approach.
Towards Safe, Realistic Testbed for Robotic Systems with Human Interaction

Simulation has been a necessary, safe testbed for robotics systems (RS). However, testing in simulation alone is not enough for robotic systems operating in close proximity, or interacting directly with, humans, because simulated humans are very limited. Furthermore, testing with real humans can be unsafe and costly. As recent advances in machine learning are being brought to physical robotic systems, how to collect data as well as evaluate them with human interactions safely yet realistically is a critical question. This paper presents a Mixed-Reality (MR) system toward human-centered development of robotic systems emphasizing benefits as a data collection and testbed tool. MR testbeds allow humans to interact with various levels of virtuality to maintain both realism and safety. We detail the advantages and limitations of these different levels of realism or virtualization, and report our MR-based RS testbed implemented using off-the-shelf MR devices with the Unity game engine and ROS. We demonstrate our testbed in a multi-robot, multi-person tracking and monitoring application. We share our vision and insights earned during the development and data collection.
Vision-based Ascending Staircase Detection with Interpretable Classification Model for Stair Climbing Robots

Robots capable of traversing flights of stairs play an important role in both indoor and outdoor applications. The capability of accurately identifying a staircase is one of the vital technical functions in these robots. This paper presents a vision-based ascending stair detection algorithm using RGB-Depth (RGB-D) data based on an interpretable model. The method follows the four steps: 1) pre-processing of RGB images for line extraction by applying the dilatation and Canny filters followed by the probabilistic Hough line transform, 2) defining the regions of interests (ROIs) via K- mean clustering, 3) training the initial model based on a support vector machine (SVM) using three extracted features (i.e., gradient, continuity factor, and deviation cost), and 4) building an interpretable model for stair classification by determining the decision boundary conditions. The developed method was evaluated for its performance using our dataset, and the results showed 85% sensitivity and 94% specificity. When the same model was tested on a different test set, the sensitivity and specificity slightly decreased to 80% and 90%, respectively. By shifting the boundary conditions using only a small subset of the new dataset without rebuilding the model, performance was improved to 90% sensitivity and 96% specificity. The presented method is also compared with existing SVM- and neural- network- based methods.
Prototype-Voxel Contrastive Learning for LiDAR Point Cloud Panoptic Segmentation

LiDAR point cloud panoptic segmentation, including both semantic and instance segmentation, plays a critical role in meticulous scene understanding for autonomous driving. Existing 3D voxelized approaches either utilize 3D sparse convolution that only focuses on local scene understanding, or add extra and time-consuming PointNet branch to capture global feature structures. To address these limitations, we propose an end-to-end Prototype-Voxel Contrastive Learning (PVCL) framework for learning stable and discriminative semantic representations, which includes voxel-level and prototype-level contrastive learning (CL). The voxel-level CL decreases intra-class distance and increases inter-class distance among sample representations, while the prototype-level CL further reduces the dependence of CL on negative sampling and avoids the influence of outliers from the same class, enabling PVCL to be more effective for outdoor point cloud panoptic segmentation. Extensive experiments are conducted on the public point cloud panoptic segmentation datasets, Semantic-KITTI and nuScenes, where evaluations and ablation studies demonstrate PVCL achieves superior performance compared with the state-of-the-art. Our approach ranks the top on the public leaderboard of Semantic-KITTI at the time of submission, and surpasses the published 2nd rank, EfficientLPS, by 1.7% in PQ.
Swin Transformer Assisted Prior Attention Network for Medical Image Segmentation

research-article Share on Swin Transformer Assisted Prior Attention Network for Medical Image Segmentation Authors: Zhihao Liao Department of Electronic Information Engineering, Nanchang University, China Department of Electronic Information Engineering, Nanchang University, ChinaSearch about this author , Kai Xu Department of Electronic Information Engineering, Nanchang University, China Department of Electronic Information Engineering, Nanchang University, ChinaSearch about this author , Neng Fan Department of Electronic Information Engineering, Nanchang University, China Department of Electronic Information Engineering, Nanchang University, ChinaSearch about this author Authors Info & Claims ICCAI '22: Proceedings of the 8th International Conference on Computing and Artificial IntelligenceMarch 2022 Pages 491–497https://doi.org/10.1145/3532213.3532287Online:13 July 2022Publication History 0citation0DownloadsMetricsTotal Citations0Total Downloads0Last 12 Months0Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access
Deep Transfer Learning Driven Oral Cancer Detection and Classification Model

Oral cancer is the most commonly occurring ‘head and neck cancers’ across the globe. Most of the oral cancer cases are diagnosed at later stages due to absence of awareness among public. Since earlier identification of disease is essential for improved outcomes, Artificial Intelligence (AI) and Machine Learning (ML) models are used in this regard. In this background, the current study introduces Artificial Intelligence with Deep Transfer Learning driven Oral Cancer detection and Classification Model (AIDTL-OCCM). The primary goal of the proposed AIDTL-OCCM model is to diagnose oral cancer using AI and image processing techniques. The proposed AIDTL-OCCM model involves fuzzy-based contrast enhancement approach to perform data pre-processing. Followed by, the densely-connected networks (DenseNet-169) model is employed to produce a useful set of deep features. Moreover, Chimp Optimization Algorithm (COA) with Autoencoder (AE) model is applied for oral cancer detection and classification. Furthermore, COA is employed to determine optimal parameters involved in AE model. A wide range of experimental analyses was conducted on benchmark datasets and the results were investigated under several aspects. The extensive experimental analysis outcomes established the enhanced performance of AIDTL-OCCM model compared to other approaches with a maximum accuracy of 90.08%.
SqueezeLight: A Multi-Operand Ring-Based Optical Neural Network With Cross-Layer Scalability

Optical neural networks (ONNs) are promising hardware platforms for next-generation artificial intelligence acceleration with ultrafast speed and low-energy consumption. However, previous ONN designs are bounded by one multiply–accumulate operation per device, showing unsatisfying scalability. In this work, we propose a scalable ONN architecture, dubbed <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SqueezeLight</monospace> . We propose a nonlinear optical neuron based on multioperand ring resonators (MORRs) to squeeze vector dot-product into a single device with low wavelength usage and built-in nonlinearity. A block-level squeezing technique with structured sparsity is exploited to support higher scalability. We adopt a robustness-aware training algorithm to guarantee variation tolerance. To enable a truly scalable ONN architecture, we extend <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SqueezeLight</monospace> to a separable optical CNN architecture that further squeezes in the layer level. Two orthogonal convolutional layers are mapped to one MORR array, leading to order-of-magnitude higher software training scalability. We further explore augmented representability for <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SqueezeLight</monospace> by introducing parametric MORR neurons with trainable nonlinearity, together with a nonlinearity-aware initialization method to stabilize convergence. Experimental results show that <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SqueezeLight</monospace> achieves one-order-of-magnitude better compactness and efficiency than previous designs with high fidelity, trainability, and robustness. Our open-source codes are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/JeremieMelo/SqueezeLight</uri> .
Blockchain Framework for Cognitive Sensor Network Using Non-Cooperative Game Theory

The incorporation of Technology in healthcare and hospital management has given a new perception to the medical procedures, and drug management for patient-centric care. Emerging technologies like blockchain, Internet of Things, and cognitive computing are most adaptable for designing smart healthcare systems. However, due to the diversified tasks involved, a design approach with broader understanding involving multiple factors that represent each area of the healthcare is the need of the hour. Game theory has gained prominence in modeling multi-player problems designated by strategic interdependency. This paper proposes a non-cooperative game strategy between players (stakeholders) to inspect how rationality is exhibited among the players who seamlessly try to get benefitted from the actions of other players. To evaluate the feasibility of the proposed model, a prototype was designed based on Ethereum permissioned blockchain employing Raspberry pi IoT devices and sensor-based cognition. Use of blockchain brings in trust, security and transparency to the system. The simulation results show that the proposed model provides better outcomes in terms of latency (&#x2248; 15 min), throughput and scalability with an increased number of transactions. The comparative analysis elucidates the fact that the proposed method outperforms the existing healthcare systems with a significant improvement of 10-15% in terms of resource utilization and provides faster and accurate patient-centric services.
End-to-End Blind Video Quality Assessment Based on Visual and Memory Attention Modeling

Developing an objective quality assessment model for user-generated content (UGC) videos is significant for multimedia applications, and also a challenge due to the diversity of video content and unpredictability of distortions. To predict the perceived quality, it is necessary to consider the human visual system, in which attention in visual and memory domains is an essential component. With the idea that the stimulus-driven bottom-up mechanism and cognition-driven top-down mechanism work in synergy to generate quality-aware attention, we propose an end-to-end blind video quality assessment (VQA) algorithm based on visual and memory attention modeling. First, a quality-aware visual attention module is established to obtain spatial-temporal attention-guided representations for frame-level quality perception. Specifically, an attention selection and confluence method is developed by circularly integrating the quality-aware attention information to spatial-temporal content features. Then, with the aid of a quality-aware memory attention module, the video-level attention-guided features are inferred through the dimension and attention reshaping of frame-level representations. The video quality is predicted with the guidance of frame-level visual attention and video-level memory attention in an end-to-end structure. Experimental results on five UGC-VQA databases (CVD2014, LIVE-Qualcomm, KoNViD-1 k, LIVE-VQC and Youtube-UGC) demonstrate the effectiveness of our modules.
Exploring the Design Context of AI-Powered Services: A Qualitative Investigation of Designers’ Experiences with Machine Learning

Artificial Intelligence (AI) has provided user experience (UX) designers with a richer toolset. To use technologies such as Machine Learning (ML) that can expand their creative capacity to design intelligent services. ML has the capability to enhance the user experience, for example, by improving efficiency, personalization, and context-aware adaptation. However, research suggests ML as a challenging design material in UX practice, such as difficulties in comprehending data dependencies when prototyping, or the lack of tools and methods for evaluating adaptive user experiences. Previous research indicates that lack of knowledge transfer into the UX design practice may hamper innovative potential. This work aims to provide new insights on how designers think about – and experience – design for AI-powered services. It is important to make ML-powered services beneficial and sustainable for end-users, organizations, and society. Therefore, we explore UX designers' reflections and experiences of using ML in a design context. We have performed nine deep explorative interviews with professional designers that work with ML. The respondents have different backgrounds, seniority, and work in different sectors. The collected interview material was qualitatively analyzed and resulted in five conceptual themes for how UX designers experience the design context surrounding AI-powered services: 1) Absence of competence, 2) Lack of incentive for competence development, 3) Challenges in articulating design criteria, 4) Mature vs. Immature clients, and 5) Lack of support for ethical concerns. We provide implications for how these themes affect the design context and practice.
A Variation-based Nighttime Image Dehazing Flow with a Physically Valid Illumination Estimator and a Luminance-Guided Coloring Model

Image quality is often reduced in hazy weather, especially during the nighttime when image visibility can be further degraded.In this paper, we propose a robust variation-based nighttime image dehazing flow with a physically valid illumination estimator, a luminance-guided coloring model and a transmission refinement procedure to effectively address this problem.We design a new illumination model to better address the non-global air-light issue in nighttime scenes.Then, we introduce a structure-preserving optimization flow based on Retinex theory to obtain ambient illumination.Color consistency is guaranteed because we use the input image as the initial guess of illumination in our coloring model.A variational procedure is developed to smoothen the estimated transmission map, where the block effect and the halos can be eliminated through the procedure.The proposed luminance-based correction mechanism further improves visual image quality in the presence of a large sky region.Our experiments are implemented based on actual hazy images.The user study indicates that the proposed method can effectively provide color consistency, preserve details, and reduce halo artifacts and noise in the resulting images compared to other state-of-the-art algorithms.When tested on real-world nighttime haze images, our dehazing flow quantitatively achieves 3.06 for NIQE and 2.99 for NR-CDIQA.
Pre-Trained Word Embedding and Language Model Improve Multimodal Machine Translation: A Case Study in Multi30K

Multimodal machine translation (MMT) is an attractive application of neural machine translation (NMT) that is commonly incorporated with image information. However, the MMT models proposed thus far have only comparable or slightly better performance than their text-only counterparts. One potential cause of this infeasibility is a lack of large-scale data. Most previous studies mitigate this limitation by employing large-scale textual parallel corpora, which are more accessible than multimodal parallel corpora, in various ways. However, these corpora are still available on only a limited scale in low-resource language pairs or domains. In this study, we leveraged monolingual (or multimodal monolingual) corpora, which are available at scale in most languages and domains, to improve MMT models. Our approach follows that of previous unimodal works that use monolingual corpora to train the word embedding or language model and incorporate them into NMT systems. While these methods demonstrated the advantage of using pre-trained representations, there is still room for MMT models to improve. To this end, our system employs debiasing procedures for the word embedding and multimodal extension of the language model (visual-language model, VLM) to make better use of the pre-trained knowledge in the MMT task. The results of evaluations conducted on the de facto MMT dataset for the English&#x2013;German translation indicate that the improvement obtained using well-tailored word embedding and VLM is approximately +1.84 BLEU and +1.63 BLEU, respectively. The evaluation on multiple language pairs reveals their adoptability across the languages. Beyond the success of our system, we also conducted an extensive analysis on VLM manipulation and showed promising areas for developing better MMT models by exploiting VLM; some benefits brought by either modality are missing, and MMT with VLM generates less fluent translations. Our code is available at https://github.com/toshohirasawa/mmt-with-monolingual-data.