Reviewerly/exHarmony huggingface dataset:
Long-tailed Recognition by Routing Diverse Distribution-Aware Experts

Natural data are often long-tail distributed over semantic classes. Existing recognition methods tackle this imbalanced classification by placing more emphasis on the tail data, through class re-balancing/re-weighting or ensembling over different data groups, resulting in increased tail accuracies but reduced head accuracies. We take a dynamic view of the training data and provide a principled model bias and variance analysis as the training data fluctuates: Existing long-tail classifiers invariably increase the model variance and the head-tail model bias gap remains large, due to more and larger confusion with hard negatives for the tail. We propose a new long-tailed classifier called RoutIng Diverse Experts (RIDE). It reduces the model variance with multiple experts, reduces the model bias with a distribution-aware diversity loss, reduces the computational cost with a dynamic expert routing module. RIDE outperforms the state-of-the-art by 5% to 7% on CIFAR100-LT, ImageNet-LT and iNaturalist 2018 benchmarks. It is also a universal framework that is applicable to various backbone networks, long-tailed algorithms, and training mechanisms for consistent performance gains. Our code is available at: https://github.com/frank-xwang/RIDE-LongTailRecognition.
Multi-Robot Task Sequencing &amp; Automatic Path Planning for Cycle Time Optimization: Application for Car Production Line

Industrial robots are programmed to repeat a sequence of well-defined tasks, e.g. deburring, cutting and welding. The production cycle time is directly influenced by the task order as well as the way trajectories linking this tasks are generated. In this letter, we present an optimization approach that minimizes the production time as well as the overall movements duration of the robots. We propose a fast algorithm that generates a sequenced near-optimal solution for Multi-Robotic Task Sequencing Problem. We model the problem in the form of a new min (sum-max) Multiple Generalized Traveling Salesman Problem `min(sum-max) MGTSP' model. Near-optimal solutions are obtained to automatically minimize cycle time in collision-free path. The originality of our method lies in its flexibility and ability to be integrated into industrial processes. In this study, we perform a double optimization in both the task and configuration space. Also, the proposed algorithm is able to automatically plan a collision-free trajectory between two robots' configurations by generating relevant via points which minimizes movements duration. Comparing to other approaches, experiment reveal positive results in terms of efficiency and demonstrate the ability of this method to be integrated into existing industrial software.
FaSTrack:A Modular Framework for Real-Time Motion Planning and Guaranteed Safe Tracking

Real-time, guaranteed safe trajectory planning is vital for navigation in unknown environments. However, real-time navigation algorithms typically sacrifice robustness for computation speed. Alternatively, provably safe trajectory planning tends to be too computationally intensive for real-time replanning. We propose FaSTrack, Fast and Safe Tracking, a framework that achieves both real-time replanning and guaranteed safety. In this framework, real-time computation is achieved by allowing any trajectory planner to use a simplified \textit{planning model} of the system. The plan is tracked by the system, represented by a more realistic, higher-dimensional \textit{tracking model}. We precompute the tracking error bound (TEB) due to mismatch between the two models and due to external disturbances. We also obtain the corresponding tracking controller used to stay within the TEB. The precomputation does not require prior knowledge of the environment. We demonstrate FaSTrack using Hamilton-Jacobi reachability for precomputation and three different real-time trajectory planners with three different tracking-planning model pairs.
Searching Images for Consensus

One of the major obstacles in reaching diagnostic consensus is observer variability. With the recent success of artificial intelligence, particularly the deep networks, the question emerges as to whether the fundamental challenge of diagnostic imaging can now be resolved. This article briefly reviews the problem and how eventually both supervised and unsupervised AI technologies could help to overcome it.
A review on type-2 fuzzy neural networks for system identification

In many engineering problems, the systems dynamics are uncertain, and then, the accurate dynamic modeling is required. Type-2 fuzzy neural networks (T2F-NNs) are extensively used in system identification problems, because of their strong estimation capability. In this paper, the application of T2F-NNs is reviewed and classified. First, an introduction to the principles of system identification, including how to extract data from a system, persistency of excitation, preprocessing of information and data, removal of outlier data, and sorting of data to learn the T2F-NNs, is presented. Then, various learning methods for structure and parameters of the T2F-NNs are reviewed and analyzed. A number of different T2F-NNs that have been used to system identification are reviewed, and their disadvantages and advantages are described. Also, their efficiency in different applications is reviewed. Finally, we will look at the horizon ahead in this issue and analyze its challenges.
PANTHER: Perception-Aware Trajectory Planner in Dynamic Environments

This paper presents PANTHER, a real-time perception-aware (PA) trajectory planner for multirotor-UAVs (Unmanned Aerial Vehicles) in dynamic environments. PANTHER plans trajectories that avoid dynamic obstacles while also keeping them in the sensor field of view (FOV) and minimizing the blur to aid in object tracking. The rotation and translation of the UAV are jointly optimized, which allows PANTHER to fully exploit the differential flatness of multirotors to maximize the PA objective. Real-time performance is achieved by implicitly imposing the underactuated dynamics of the UAV through the Hopf fibration. PANTHER is able to keep the obstacles inside the FOV 7.9 and 1.5 times more than non-PA approaches and PA approaches that decouple translation and yaw, respectively. The projected velocity (and hence the blur) is reduced by 18&#x0025; and 34&#x0025;, respectively. This leads to average success rates three times larger than state-of-the-art approaches in multi-obstacle avoidance scenarios. The MINVO basis is used to impose low-conservative collision avoidance constraints in position and velocity space. Finally, extensive hardware experiments in unknown dynamic environments with all the computation running onboard are presented, with velocities of up to 5.8 m/s, and with relative velocities (with respect to the obstacles) of up to 6.3 m/s. The only sensors used are an IMU, a forward-facing depth camera, and a downward-facing monocular camera.
Temporal pointwise convolutional networks for length of stay prediction in the intensive care unit

The pressure of ever-increasing patient demand and budget restrictions make hospital bed management a daily challenge for clinical staff. Most critical is the efficient allocation of resource-heavy Intensive Care Unit (ICU) beds to the patients who need life support. Central to solving this problem is knowing for how long the current set of ICU patients are likely to stay in the unit. In this work, we propose a new deep learning model based on the combination of temporal convolution and pointwise (1x1) convolution, to solve the length of stay prediction task on the eICU and MIMIC-IV critical care datasets. The model - which we refer to as Temporal Pointwise Convolution (TPC) - is specifically designed to mitigate common challenges with Electronic Health Records, such as skewness, irregular sampling and missing data. In doing so, we have achieved significant performance benefits of 18-68% (metric and dataset dependent) over the commonly used Long-Short Term Memory (LSTM) network, and the multi-head self-attention network known as the Transformer. By adding mortality prediction as a side-task, we can improve performance further still, resulting in a mean absolute deviation of 1.55 days (eICU) and 2.28 days (MIMIC-IV) on predicting remaining length of stay.
A novel image encryption based on rossler map diffusion and particle swarm optimization generated highly non-linear substitution boxes

Symmetric encryption is appraised as one of the key ways in which end-to-end data transfer security is guaranteed. To inject confusion in the substitution phase of the modern block encryption system, substitution boxes are utilized. The design of the S-box possesses a high influence on the strength and sturdiness of modern block encryption systems. In this document, we propose to introduce an efficient methodology of creating highly non-linear cryptographic substitution boxes as an alternate to chaotic, or algebraic construction methods. Particle Swarm Optimization is utilized in the construction of highly non-linear S-boxes, in the projected technique the initial population is randomly produced, and the position vector of particles is used in generating S-boxes. Performance appraisal of the constructed S-boxes is confirmed by standard criteria. To assess their appropriateness and their application for encryption, an image encryption scheme of the projected S-boxes is correspondingly suggested, the proposed cryptosystem is evaluated against different standard security analysis tests. The results show that the Proposed S-boxes based cryptosystem bearing strong immunity against various cryptographic attacks.
Research and Case Analysis of Apriori Algorithm Based on Mining Frequent Item-Sets

In the background of the information age, the importance of data resources can be imagined, and the importance of the use of data resources meansdata mining has also emerged.In the current situation, all industries are in a relatively equal stage, should make good use of data resources, use Apriori algorithm to mine association rules, formulate marketing strategies, promote sales growth and slow down the loss of national GDP.Countries can also use data mining to make predictions and support decisions.Briefly describe the basic concepts of data mining and association rules.Only with a basic understanding of data mining and association rules can we better understand the Apriori algorithm, why it is one of the most classic and influential algorithms.Understanding its essence and function, and suggesting suggestions for improvement by analyzing the ideas of the Apriori algorithm, understanding its process, advantages and disadvantages, and through the data collected from a supermarket in Bozhou City, Anhui Province, China; and understanding the application of the algorithm in the field of e-commerce and universities, and the effect it will achieve.
Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks

Attention mechanisms, especially self-attention, have played an increasingly important role in deep feature representation for visual tasks. Self-attention updates the feature at each position by computing a weighted sum of features using pair-wise affinities across all positions to capture the long-range dependency within a single sample. However, self-attention has quadratic complexity and ignores potential correlation between different samples. This paper proposes a novel attention mechanism which we call external attention, based on two external, small, learnable, shared memories, which can be implemented easily by simply using two cascaded linear layers and two normalization layers; it conveniently replaces self-attention in existing popular architectures. External attention has linear complexity and implicitly considers the correlations between all data samples. We further incorporate the multi-head mechanism into external attention to provide an all-MLP architecture, external attention MLP (EAMLP), for image classification. Extensive experiments on image classification, object detection, semantic segmentation, instance segmentation, image generation, and point cloud analysis reveal that our method provides results comparable or superior to the self-attention mechanism and some of its variants, with much lower computational and memory costs.
Deep-Masking Generative Network: A Unified Framework for Background Restoration From Superimposed Images

Restoring the clean background from the superimposed images containing a noisy layer is the common crux of a classical category of tasks on image restoration such as image reflection removal, image deraining and image dehazing. These tasks are typically formulated and tackled individually due to the diverse and complicated appearance patterns of noise layers within the image. In this work we present the Deep-Masking Generative Network (DMGN), which is a unified framework for background restoration from the superimposed images and is able to cope with different types of noise. Our proposed DMGN follows a coarse-to-fine generative process: a coarse background image and a noise image are first generated in parallel, then the noise image is further leveraged to refine the background image to achieve a higher-quality background image. In particular, we design the novel Residual Deep-Masking Cell as the core operating unit for our DMGN to enhance the effective information and suppress the negative information during image generation via learning a gating mask to control the information flow. By iteratively employing this Residual Deep-Masking Cell, our proposed DMGN is able to generate both high-quality background image and noisy image progressively. Furthermore, we propose a two-pronged strategy to effectively leverage the generated noise image as contrasting cues to facilitate the refinement of the background image. Extensive experiments across three typical tasks for image background restoration, including image reflection removal, image rain steak removal and image dehazing, show that our DMGN consistently outperforms state-of-the-art methods specifically designed for each single task.
Coverage path planning for spraying drones

The pandemic by COVID-19 is causing a devastating effect on the health of the global population. Currently, there are several efforts to prevent the spread of the virus. Among those efforts, cleaning and disinfecting public areas have become important tasks and they should be automated in future smart cities. To contribute in this direction, this paper proposes a coverage path planning method for a spraying drone, an unmanned aerial vehicle that has mounted a sprayer/sprinkler system, that can disinfect areas. State-of-the-art planners consider a camera instead of a sprinkler, in consequence, the expected coverage will differ in running time because the liquid dispersion is different from a camera's projection model. In addition, current planners assume that the vehicles can fly outside the target region; this assumption can not be satisfied in our problem, because disinfections are performed at low altitudes. Our method presents i) a new sprayer/sprinkler model that fits a more realistic coverage volume to the drop dispersion and ii) a planning method that efficiently restricts the flight to the region of interest avoiding potential collisions in bounded scenes. The algorithm has been tested in several simulation scenes, showing that it is effective and covers more areas with respect to two approaches in the literature. Note that the proposal is not limited to disinfection applications, but can be applied to other ones, such as painting or precision agriculture.
Patient-specific COVID-19 resource utilization prediction using fusion AI model

Abstract The strain on healthcare resources brought forth by the recent COVID-19 pandemic has highlighted the need for efficient resource planning and allocation through the prediction of future consumption. Machine learning can predict resource utilization such as the need for hospitalization based on past medical data stored in electronic medical records (EMR). We conducted this study on 3194 patients (46% male with mean age 56.7 (±16.8), 56% African American, 7% Hispanic) flagged as COVID-19 positive cases in 12 centers under Emory Healthcare network from February 2020 to September 2020, to assess whether a COVID-19 positive patient’s need for hospitalization can be predicted at the time of RT-PCR test using the EMR data prior to the test. Five main modalities of EMR, i.e., demographics, medication, past medical procedures, comorbidities, and laboratory results, were used as features for predictive modeling, both individually and fused together using late, middle, and early fusion. Models were evaluated in terms of precision, recall, F1-score (within 95% confidence interval). The early fusion model is the most effective predictor with 84% overall F1-score [CI 82.1–86.1]. The predictive performance of the model drops by 6 % when using recent clinical data while omitting the long-term medical history. Feature importance analysis indicates that history of cardiovascular disease, emergency room visits in the past year prior to testing, and demographic factors are predictive of the disease trajectory. We conclude that fusion modeling using medical history and current treatment data can forecast the need for hospitalization for patients infected with COVID-19 at the time of the RT-PCR test.
On lower iteration complexity bounds for the convex concave saddle point problems

In this paper, we study the lower iteration complexity bounds for finding the saddle point of a strongly convex and strongly concave saddle point problem: $\min_x\max_yF(x,y)$. We restrict the classes of algorithms in our investigation to be either pure first-order methods or methods using proximal mappings. The existing lower bound result for this type of problems is obtained via the framework of strongly monotone variational inequality problems, which corresponds to the case where the gradient Lipschitz constants ($L_x, L_y$ and $L_{xy}$) and strong convexity/concavity constants ($\mu_x$ and $\mu_y$) are uniform with respect to variables $x$ and $y$. However, specific to the min-max saddle point problem these parameters are naturally different. Therefore, one is led to finding the best possible lower iteration complexity bounds, specific to the min-max saddle point models. In this paper we present the following results. For the class of pure first-order algorithms, our lower iteration complexity bound is $\Omega\left(\sqrt{\frac{L_x}{\mu_x}+\frac{L_{xy}^2}{\mu_x\mu_y}+\frac{L_y}{\mu_y}}\cdot\ln\left(\frac{1}{\epsilon}\right)\right)$, where the term $\frac{L_{xy}^2}{\mu_x\mu_y}$ explains how the coupling influences the iteration complexity. Under several special parameter regimes, this lower bound has been achieved by corresponding optimal algorithms. However, whether or not the bound under the general parameter regime is optimal remains open. Additionally, for the special case of bilinear coupling problems, given the availability of certain proximal operators, a lower bound of $\Omega\left(\sqrt{\frac{L_{xy}^2}{\mu_x\mu_y}+1}\cdot\ln(\frac{1}{\epsilon})\right)$ is established in this paper, and optimal algorithms have already been developed in the literature.
ShadowGNN: Graph Projection Neural Network for Text-to-SQL Parser

Given a database schema, Text-to-SQL aims to translate a natural language question into the corresponding SQL query.Under the setup of cross-domain, traditional semantic parsing models struggle to adapt to unseen database schemas.To improve the model generalization capability for rare and unseen schemas, we propose a new architecture, ShadowGNN, which processes schemas at abstract and semantic levels.By ignoring names of semantic items in databases, abstract schemas are exploited in a well-designed graph projection neural network to obtain delexicalized representation of question and schema.Based on the domain-independent representations, a relation-aware transformer is utilized to further extract logical linking between question and schema.Finally, a SQL decoder with context-free grammar is applied.On the challenging Text-to-SQL benchmark Spider, empirical results show that ShadowGNN outperforms state-of-the-art models.When the annotated data is extremely limited (only 10% training set), ShadowGNN gets over absolute 5% performance gain, which shows its powerful generalization ability.Our implementation will be open-sourced at https://github.com/WowCZ/shadowgnn.
A survey on the blockchain techniques for the Internet of Vehicles security

Abstract Recently, The Internet of Vehicles (IoV) concept is becoming very popular due to sharing of the data between vehicles and the infrastructure. The sharing of data is very important for enhancing vehicular services, but at the same time makes IoV vulnerable for security and privacy issues. The smart and interconnected vehicles produce sophisticated services for transport authorities, car manufacturers, vehicle owners, and other service providers. IoV are very vulnerable to malicious attacks due to its self‐organizing nature and the open source nature of its implementations. This exposes the smart and interconnected vehicles to a variety of privacy and security threats, such as a remote hijacking or location tracking of vehicles. Thus, the security for IoV environment is critical. Blockchain technology has been recently used for cybersecurity due to the robustness and integrity preserving nature of its design. This review article provides a detailed survey of existing work in the literature to secure IoV through blockchain techniques such as security, privacy, reputation, distributed, decentralized, data sharing, authentication, and trust‐based approaches. The paper presents the detailed discussion and analysis of these blockchain techniques to secure IoV. In addition, we present the gaps and research challenges identified from the existing research works. This provides work directions for future research in blockchain techniques to secure IoV.
INVIGORATE: Interactive Visual Grounding and Grasping in Clutter

This paper presents INVIGORATE, a robot system that interacts with human through natural language and grasps a specified object in clutter.The objects may occlude, obstruct, or even stack on top of one another.INVIGORATE embodies several challenges: (i) infer the target object among other occluding objects, from input language expressions and RGB images, (ii) infer object blocking relationships (OBRs) from the images, and (iii) synthesize a multi-step plan to ask questions that disambiguate the target object and to grasp it successfully.We train separate neural networks for object detection, for visual grounding, for question generation, and for OBR detection and grasping.They allow for unrestricted object categories and language expressions, subject to the training datasets.However, errors in visual perception and ambiguity in human languages are inevitable and negatively impact the robot's performance.To overcome these uncertainties, we build a partially observable Markov decision process (POMDP) that integrates the learned neural network modules.Through approximate POMDP planning, the robot tracks the history of observations and asks disambiguation questions in order to achieve a near-optimal sequence of actions that identify and grasp the target object.INVIGORATE combines the benefits of model-based POMDP planning and data-driven deep learning.Preliminary experiments with INVIGORATE on a Fetch robot show significant benefits of this integrated approach to object grasping in clutter with natural language interactions.A demonstration video is available online 1 .