Reviewerly/exHarmony huggingface dataset:
Lift, Splat, Shoot: Encoding Images from Arbitrary Camera Rigs by Implicitly Unprojecting to 3D

The goal of perception for autonomous vehicles is to extract semantic representations from multiple sensors and fuse these representations into a single “bird’s-eye-view” coordinate frame for consumption by motion planning. We propose a new end-to-end architecture that directly extracts a bird’s-eye-view representation of a scene given image data from an arbitrary number of cameras. The core idea behind our approach is to “lift” each image individually into a frustum of features for each camera, then “splat” all frustums into a rasterized bird’s-eye-view grid. By training on the entire camera rig, we provide evidence that our model is able to learn not only how to represent images but how to fuse predictions from all cameras into a single cohesive representation of the scene while being robust to calibration error. On standard bird’s-eye-view tasks such as object segmentation and map segmentation, our model outperforms all baselines and prior work. In pursuit of the goal of learning dense representations for motion planning, we show that the representations inferred by our model enable interpretable end-to-end motion planning by “shooting” template trajectories into a bird’s-eye-view cost map output by our network. We benchmark our approach against models that use oracle depth from lidar. Project page with code: https://nv-tlabs.github.io/lift-splat-shoot .
Knowledge Distillation Meets Self-supervision

Knowledge distillation, which involves extracting the “dark knowledge” from a teacher network to guide the learning of a student network, has emerged as an important technique for model compression and transfer learning. Unlike previous works that exploit architecture-specific cues such as activation and attention for distillation, here we wish to explore a more general and model-agnostic approach for extracting “richer dark knowledge” from the pre-trained teacher model. We show that the seemingly different self-supervision task can serve as a simple yet powerful solution. For example, when performing contrastive learning between transformed entities, the noisy predictions of the teacher network reflect its intrinsic composition of semantic and pose information. By exploiting the similarity between those self-supervision signals as an auxiliary task, one can effectively transfer the hidden information from the teacher to the student. In this paper, we discuss practical ways to exploit those noisy self-supervision signals with selective transfer for distillation. We further show that self-supervision signals improve conventional distillation with substantial gains under few-shot and noisy-label scenarios. Given the richer knowledge mined from self-supervision, our knowledge distillation approach achieves state-of-the-art performance on standard benchmarks, i.e., CIFAR100 and ImageNet, under both similar-architecture and cross-architecture settings. The advantage is even more pronounced under the cross-architecture setting, where our method outperforms the state of the art by an average of 2.3% in accuracy rate on CIFAR100 across six different teacher-student pairs. The code and models are available at: https://github.com/xuguodong03/SSKD .
Monarch butterfly optimization: A comprehensive review

Swarm intelligence (SI) is the collective behavior of decentralized, self-organized natural or artificial systems. Monarch butterfly optimization (MBO) algorithm is a class of swarm intelligence metaheuristic algorithm inspired by the migration behavior of monarch butterflies. Through the migration operation and butterfly adjusting operation, individuals in MBO are updated. MBO can outperform many state-of-the-art optimization techniques when solving global numerical optimization and engineering problems. This paper presents a comprehensive review of the MBO algorithm including its modifications, hybridizations, variants, and applications. Additionally, further research directions for MBO are discussed. This review study serves as a solid reference for future studies in the arena of SI and in particular the MBO algorithm.
Bilateral Attention Network for RGB-D Salient Object Detection

Most existing RGB-D salient object detection (SOD) methods focus on the foreground region when utilizing the depth images. However, the background also provides important information in traditional SOD methods for promising performance. To better explore salient information in both foreground and background regions, this paper proposes a Bilateral Attention Network (BiANet) for the RGB-D SOD task. Specifically, we introduce a Bilateral Attention Module (BAM) with a complementary attention mechanism: foreground-first (FF) attention and background-first (BF) attention. The FF attention focuses on the foreground region with a gradual refinement style, while the BF one recovers potentially useful salient information in the background region. Benefitted from the proposed BAM module, our BiANet can capture more meaningful foreground and background cues, and shift more attention to refining the uncertain details between foreground and background regions. Additionally, we extend our BAM by leveraging the multi-scale techniques for better SOD performance. Extensive experiments on six benchmark datasets demonstrate that our BiANet outperforms other state-of-the-art RGB-D SOD methods in terms of objective metrics and subjective visual comparison. Our BiANet can run up to 80fps on $224\times224$ RGB-D images, with an NVIDIA GeForce RTX 2080Ti GPU. Comprehensive ablation studies also validate our contributions.
Privacy concerns can explain unwillingness to download and use contact tracing apps when COVID-19 concerns are high

Governments around the world have increasingly relied on technology to promote public health, such as using electronic health records and portable fitness devices. During the COVID-19 pandemic, "contact tracing apps" for smartphones have also been promoted in many countries as a way to allow public officials to facilitate contact tracing. But uptake in virtually all countries where such apps have been promoted is slow, one reason being privacy concerns. Conducting three experiments across France (n = 471), Australia (n = 202), and the United States (n = 1005), we explore if salient COVID-19 concerns, which intuitively should increase concerns about personal and public health, might in fact increase privacy concerns and thereby reduce uptake of contact tracing apps. Using an experimental design where we randomly assign participants to either a disease concerns or control condition, we find that salient COVID-19 concerns decrease intentions to download contact tracing apps. Mediation results reveal that greater valuations of privacy explain the lower willingness. We therefore explain why COVID-19 contact tracing apps that are promoted when the pandemic is at its peak see low levels of uptake. Our results provide policy makers with implications concerning how to promote uptake to help "flatten the curve" of not just the current pandemic but potentially also future ones.
EfficientPS: Efficient Panoptic Segmentation

Abstract Understanding the scene in which an autonomous robot operates is critical for its competent functioning. Such scene comprehension necessitates recognizing instances of traffic participants along with general scene semantics which can be effectively addressed by the panoptic segmentation task. In this paper, we introduce the Efficient Panoptic Segmentation (EfficientPS) architecture that consists of a shared backbone which efficiently encodes and fuses semantically rich multi-scale features. We incorporate a new semantic head that aggregates fine and contextual features coherently and a new variant of Mask R-CNN as the instance head. We also propose a novel panoptic fusion module that congruously integrates the output logits from both the heads of our EfficientPS architecture to yield the final panoptic segmentation output. Additionally, we introduce the KITTI panoptic segmentation dataset that contains panoptic annotations for the popularly challenging KITTI benchmark. Extensive evaluations on Cityscapes, KITTI, Mapillary Vistas and Indian Driving Dataset demonstrate that our proposed architecture consistently sets the new state-of-the-art on all these four benchmarks while being the most efficient and fast panoptic segmentation architecture to date.
6G networks: Beyond Shannon towards semantic and goal-oriented communications

The goal of this paper is to promote the idea that including semantic and goal-oriented aspects in future 6G networks can produce a significant leap forward in terms of system effectiveness and sustainability. Semantic communication goes beyond the common Shannon paradigm of guaranteeing the correct reception of each single transmitted bit, irrespective of the meaning conveyed by the transmitted bits. The idea is that, whenever communication occurs to convey meaning or to accomplish a goal, what really matters is the impact that the received bits have on the interpretation of the meaning intended by the transmitter or on the accomplishment of a common goal. Focusing on semantic and goal-oriented aspects, and possibly combining them, helps to identify the relevant information, i.e. the information strictly necessary to recover the meaning intended by the transmitter or to accomplish a goal. Combining knowledge representation and reasoning tools with machine learning algorithms paves the way to build semantic learning strategies enabling current machine learning algorithms to achieve better interpretation capabilities and contrast adversarial attacks. 6G semantic networks can bring semantic learning mechanisms at the edge of the network and, at the same time, semantic learning can help 6G networks to improve their efficiency and sustainability.
Sentiment analysis on the impact of coronavirus in social life using the BERT model

Nowadays, the whole world is confronting an infectious disease called the coronavirus. No country remained untouched during this pandemic situation. Due to no exact treatment available, the disease has become a matter of seriousness for both the government and the public. As social distance is considered the most effective way to stay away from this disease. Therefore, to address the people eagerness about the Corona pandemic and to express their views, the trend of people has moved very fast towards social media. Twitter has emerged as one of the most popular platforms among those social media platforms. By studying the same eagerness and opinions of people to understand their mental state, we have done sentiment analysis using the BERT model on tweets. In this paper, we perform a sentiment analysis on two data sets; one data set is collected by tweets made by people from all over the world, and the other data set contains the tweets made by people of India. We have validated the accuracy of the emotion classification from the GitHub repository. The experimental results show that the validation accuracy is ≈ 94%.
Calibration of fringe projection profilometry: A comparative review

Fringe projection profilometry is widely used for three-dimensional (3D) measurements. System calibration is crucial to fringe projection systems, since it determines how the phase is converted into 3D geometry. Many methods and studies on calibration have been presented in the last few years. However, a comparative overview on the accuracy and the implementation details is still lacking. In this work, we present a review of some of the most used calibration methods. First, the basic principles are reviewed. Then, both quantitative and qualitative comparative experiments are implemented to show the advantages and constraints of each method. Further, a number of factors that may affect the performance of the calibration are discussed. Sample codes are available for better comprehension and a quick start of these methods.
Approximating XGBoost with an interpretable decision tree

The increasing usage of machine-learning models in critical domains has recently stressed the necessity of interpretable machine-learning models. In areas like healthcare, finary – the model consumer must understand the rationale behind the model output in order to use it when making a decision. For this reason, it is impossible to use black-box models in these scenarios, regardless of their high predictive performance. Decision forests, and in particular Gradient Boosting Decision Trees (GBDT), are examples of this kind of model. GBDT models are considered the state-of-the-art in many classification challenges, reflected by the fact that the majority of Kaggle’s recent winners used GBDT methods as a part of their solution (such as XGBoost). But despite their superior predictive performance, they cannot be used in tasks that require transparency. This paper presents a novel method for transforming a decision forest of any kind into an interpretable decision tree. The method extends the tool-set available for machine learning practitioners, who want to exploit the interpretability of decision trees without significantly impairing the predictive performance gained by GBDT models like XGBoost. We show in an empirical evaluation that in some cases the generated tree is able to approximate the predictive performance of a XGBoost model while enabling better transparency of the outputs.
BiERU: Bidirectional emotional recurrent unit for conversational sentiment analysis

Sentiment analysis in conversations has gained increasing attention in recent years for the growing amount of applications it can serve, e.g., sentiment analysis, recommender systems, and human-robot interaction. The main difference between conversational sentiment analysis and single sentence sentiment analysis is the existence of context information that may influence the sentiment of an utterance in a dialogue. How to effectively encode contextual information in dialogues, however, remains a challenge. Existing approaches employ complicated deep learning structures to distinguish different parties in a conversation and then model the context information. In this paper, we propose a fast, compact and parameter-efficient party-ignorant framework named bidirectional emotional recurrent unit for conversational sentiment analysis. In our system, a generalized neural tensor block followed by a two-channel classifier is designed to perform context compositionality and sentiment classification, respectively. Extensive experiments on three standard datasets demonstrate that our model outperforms the state of the art in most cases.
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models

Deep generative models are a class of techniques that train deep neural networks to model the distribution of training samples. Research has fragmented into various interconnected approaches, each of which make trade-offs including run-time, diversity, and architectural restrictions. In particular, this compendium covers energy-based models, variational autoencoders, generative adversarial networks, autoregressive models, normalizing flows, in addition to numerous hybrid approaches. These techniques are compared and contrasted, explaining the premises behind each and how they are interrelated, while reviewing current state-of-the-art advances and implementations.
Northern Goshawk Optimization: A New Swarm-Based Algorithm for Solving Optimization Problems

Optimization algorithms are one of the effective stochastic methods in solving optimization problems. In this paper, a new swarm-based algorithm called Northern Goshawk Optimization (NGO) algorithm is presented that simulates the behavior of northern goshawk during prey hunting. This hunting strategy includes two phases of prey identification and the tail and chase process. The various steps of the proposed NGO algorithm are described and then its mathematical modeling is presented for use in solving optimization problems. The ability of NGO to solve optimization problems is evaluated on sixty-eight different objective functions. To analyze the quality of the results, the proposed NGO algorithm is compared with eight well-known algorithms, particle swarm optimization, genetic algorithm, teaching-learning based optimization, gravitational search algorithm, grey wolf optimizer, whale optimization algorithm, tunicate swarm algorithm, and marine predators algorithm. In addition, for further analysis, the proposed algorithm is also employed to solve four engineering design problems. The results of simulations and experiments show that the proposed NGO algorithm, by creating a proper balance between exploration and exploitation, has an effective performance in solving optimization problems and is much more competitive than similar algorithms.
A comprehensive survey of arabic sentiment analysis

Sentiment analysis (SA) is a continuing field of research that lies at the intersection of many fields such as data mining, natural language processing and machine learning. It is concerned with the automatic extraction of opinions conveyed in a certain text. Due to its vast applications, many studies have been conducted in the area of SA especially on English texts, while other languages such as Arabic received less attention. This survey presents a comprehensive overview of the works done so far on Arabic SA (ASA). The survey groups published papers based on the SA-related problems they address and tries to identify the gaps in the current literature laying foundation for future studies in this field.
Activity Recognition Using Temporal Optical Flow Convolutional Features and Multilayer LSTM

Nowadays digital surveillance systems are universally installed for continuously collecting enormous amounts of data, thereby requiring human monitoring for the identification of different activities and events. Smarter surveillance is the need of this era through which normal and abnormal activities can be automatically identified using artificial intelligence and computer vision technology. In this paper, we propose a framework for activity recognition in surveillance videos captured over industrial systems. The continuous surveillance video stream is first divided into important shots, where shots are selected using the proposed convolutional neural network (CNN) based human saliency features. Next, temporal features of an activity in the sequence of frames are extracted by utilizing the convolutional layers of a FlowNet2 CNN model. Finally, a multilayer long short-term memory is presented for learning long-term sequences in the temporal optical flow features for activity recognition. Experiments <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">11</sup> https://github.com/Aminullah6264/Activity_Rec_ML-LSTM. are conducted using different benchmark action and activity recognition datasets, and the results reveal the effectiveness of the proposed method for activity recognition in industrial settings compared with state-of-the-art methods.
Index-based permutation-diffusion in multiple-image encryption using DNA sequence

To improve the encryption quality and increase the speed of transmission over the internet, this study aims to provide a fast secure multiple-image encryption (MIE) algorithm based on DNA sequence and image matrix indexes. Because multiple images are considered in the proposed MIE algorithm, one big concern refers to the speed of the algorithm. In the first phase of the proposed method, multiple plain- images are attached together to create a single image. Next, this image is converted to one-dimension array. Half of the array indexes are used to permute all the pixels position. During the permutation, the same indexes are associated with DNA sequence to diffuse the pixels gray level. Simulation results demonstrate that using half of indexes for permutation and diffusion make the proposed algorithm very fast and also using DNA sequence encoding gives the algorithm enough power to resist against common attacks in the era of image encryption.
A privacy-preserving cryptosystem for IoT E-healthcare

Privacy preservation has become a prerequisite for modern applications in the cloud, social media, Internet of things (IoT), and E- healthcare systems. In general, health and medical data contain images and medical information about the patients and such personal data should be kept confidential in order to maintain the patients’ privacy. Due to limitations in digital data properties, traditional encryption schemes over textual and structural one-dimension data cannot be applied directly to e-health data. In addition, when personal data are sent over the open channels, patients may lose privacy of data contents. Hence, a secure lightweight keyframe extraction method is highly required to ensure timely, correct, and privacy-preserving e-health services. Besides this, it is inherently difficult to achieve a satisfied level of security in a cost-effective way while considering the constraints of real-time e-health applications. In this paper, we propose a privacy preserving chaos-based encryption cryptosystem for patients’ privacy protection. The proposed cryptosystem can protect patient’s images from a compromised broker. In particular, we propose a fast probabilistic cryptosystem to secure medical keyframes that are extracted from wireless capsule endoscopy procedure using a prioritization method. The encrypted images produced by our cryptosystem exhibits randomness behavior, which guarantee computational efficiency as well as a highest level of security for the keyframes against various attacks. Furthermore, it processes the medical data without leaking any information, thus preserving patient’s privacy by allowing only authorized users for decryption. The experimental results and security analysis from different perspectives verify the excellent performance of our encryption cryptosystem compared to other recent encryption schemes.
Multiple Feature Reweight DenseNet for Image Classification

Recent network research has demonstrated that the performance of convolutional neural networks can be improved by introducing a learning block that captures spatial correlations.In this paper, we propose a novel multiple feature reweight DenseNet (MFR-DenseNet) architecture.The MFR-DenseNet improves the representation power of the DenseNet by adaptively recalibrating the channel-wise feature responses and explicitly modeling the interdependencies between the features of different convolutional layers.First, in order to perform dynamic channel-wise feature recalibration, we construct the channel feature reweight DenseNet (CFR-DenseNet) by introducing the squeeze-and-excitation module (SEM) to DenseNet.Then, to model the interdependencies between the features of different convolutional layers, we propose the double squeeze-and-excitation module (DSEM) and construct the inter-layer feature reweight DenseNet (ILFR-DenseNet).In the last step, we designed the MFR-DenseNet by combining the CFR-DenseNet and the ILFR-DenseNet with an ensemble learning approach.Our experiments demonstrate the effectiveness of CFR-DenseNet, ILFR-DenseNet, and MFR-DenseNet.More importantly, the MFR-DenseNet drops the error rate on CIFAR-10 and CIFAR-100 by a large margin with significantly fewer parameters.Our 100-layer MFR-DenseNet (with 7.1M parameters) model achieves competitive results on CIFAR-10 and CIFAR-100 data sets, with test errors of 3.57% and 18.27% respectively, achieving a 4.5% relative improvement on CIFAR-10 and a 5.09% relative improvement on CIFAR-100 over the best result of DenseNet (with 27.2M parameters).