Reviewerly/exHarmony huggingface dataset:
Open Set Incremental Learning for Automatic Target Recognition

Incremental learning methods update the existing model with new knowledge when the target data increase continuously. Open set recognition (OSR) algorithms provide classifiers with a rejection option so that the new untrained target type is identified. In this paper, an open set incremental learning method is introduced for automatic target recognition, which is able to recognize and learn the new unknown classes continually. The proposed method, open set model with incremental learning (OSmIL), is an ensemble classifier so it is able to be updated only by the new data. For saving the computational time and storage source, a new exemplar selection method is introduced for model simplifying. Edge samples are selected to cover training classes; as a result, the model size is deduced and controlled. Moreover, because extreme value theory (EVT) is suitable to fit a classification model that includes open space risk, the decision function based on EVT makes an open set classifier for identifying the new classes. Experimental results demonstrate that the proposed OSmIL outperforms the other state of the arts on the accuracy of multiclass OSR. And OSmIL can maintain good accuracy and efficiency in the incremental learning experiment set.
Automated oral squamous cell carcinoma identification using shape, texture and color features of whole image strips

Despite profound knowledge of the incidence of oral cancers and a large body of research beyond it, it continues to beat diagnosis and treatment management. Post physical observation by clinicians, a biopsy is a gold standard for accurate detection of any abnormalities. Towards the application of artificial intelligence as an aid to diagnosis, automated cell nuclei segmentation is the most essential step for the recognition of the cancer cells. In this study, we have extracted the shape, texture and color features from the histopathological images collected indigenously from regional hospitals. A dataset of 42 whole slide slices was used to automatically segment and generate a cell level dataset of 720 nuclei. Next, different classifiers were applied for classification purposes. 99.4 % accuracy using Decision Tree Classifier, 100 % accuracy using both SVM and Logistic regression and 100 % accuracy using SVM, Logistic regression and Linear Discriminant were acquired for shape, textural and color features respectively. The in-depth analysis showed SVM and Linear Discriminant classifier gave the best result for texture and color features respectively. The achieved result can be effectively converted to software as an assistant diagnostic tool.
Channel Attention Module with Multi-scale Grid Average Pooling for Breast Cancer Segmentation in an Ultrasound Image

Breast cancer accounts for the second-largest number of deaths in women around the world, and more than 8% of women will suffer from the disease in their lifetime. Mortality due to breast cancer can be reduced by its early and precise diagnosis. Many studies have investigated methods for segmentation, and computer-aided diagnosis based on deep learning techniques, in particular, has recently gained attention. However, recently proposed methods such as fully convolutional network (FCN), SegNet, and U-Net still need to be further improved to provide better semantic segmentation when diagnosing breast cancer by ultrasound imaging, because of their low performance. In this article, we propose a channel attention module with multiscale grid average pooling (MSGRAP) for the precise segmentation of breast cancer regions in ultrasound images. We demonstrate the effectiveness of the channel attention module with MSGRAP for semantic segmentation and develop a novel semantic segmentation network with the proposed attention module for the precise segmentation of breast cancer regions in ultrasound images. While a conventional convolutional operation cannot use global spatial information on input images and only use the small local information in a kernel of a convolution filter, the proposed attention module allows using both global and local spatial information. In addition, through ablation studies, we come up with a network architecture for precise breast cancer segmentation in an ultrasound image. The proposed network was constructed with an open-source breast cancer ultrasound image data set, and its performance was compared with those of other state-of-the-art deep-learning models for the segmentation of breast cancer. The experimental results showed that our network outperformed other segmentation methods, and the proposed channel attention module improved the performance of the network for breast cancer segmentation in ultrasound images.
Emojis influence emotional communication, social attributions, and information processing

Many emojis symbolize nonverbal cues that are used during face-to-face communication. Despite their popularity, few studies have examined how emojis influence digital interactions. The present study addresses this gap by measuring the impact of emojis on emotion interpretation, social attributions, and information processing. Participants read messages that are typical of social exchanges in instant text messaging (IM) accompanied by emojis that mimic negative, positive and neutral facial expressions. Sentence valence and emoji valence were paired in a fully crossed design such that verbal and nonverbal messages were either congruent or incongruent. Perceived emotional state of the sender, perceived warmth, and patterns of eye movements that reflect information processing were measured. A negativity effect was observed whereby the sender's mood was perceived as negative when a negative emoji and/or a negative sentence were presented. Moreover, the presence of a negative emoji intensified the perceived negativity of negative sentences. Adding a positive emoji to a message increased the perceived warmth of the sender. Finally, processing speed and understanding of verbal messages was enhanced by the presence of congruent emojis. Our results therefore support the use of emojis, and in particular positive emojis, to improve communication, express feelings, and make a positive impression during socially-driven digital interactions.
An Efficient Anomaly Recognition Framework Using an Attention Residual LSTM in Surveillance Videos

Video anomaly recognition in smart cities is an important computer vision task that plays a vital role in smart surveillance and public safety but is challenging due to its diverse, complex, and infrequent occurrence in real-time surveillance environments. Various deep learning models use significant amounts of training data without generalization abilities and with huge time complexity. To overcome these problems, in the current work, we present an efficient light-weight convolutional neural network (CNN)-based anomaly recognition framework that is functional in a surveillance environment with reduced time complexity. We extract spatial CNN features from a series of video frames and feed them to the proposed residual attention-based long short-term memory (LSTM) network, which can precisely recognize anomalous activity in surveillance videos. The representative CNN features with the residual blocks concept in LSTM for sequence learning prove to be effective for anomaly detection and recognition, validating our model’s effective usage in smart cities video surveillance. Extensive experiments on the real-world benchmark UCF-Crime dataset validate the effectiveness of the proposed model within complex surveillance environments and demonstrate that our proposed model outperforms state-of-the-art models with a 1.77%, 0.76%, and 8.62% increase in accuracy on the UCF-Crime, UMN and Avenue datasets, respectively.
Bearing defect identification by swarm decomposition considering permutation entropy measure and opposition-based slime mould algorithm

An intelligent defect identification scheme has been proposed to identify the taper roller bearing defects through the extreme learning machine (ELM) model. The raw vibration signal from the bearing test rig is decomposed into different modes by Swarm decomposition (SWD) method to remove noise. The permutation entropy (PE) is taken as a measurement index to select the prominent mode. Features sensitive to defect conditions are extracted from prominent mode using a filter-based relief algorithm. The ranking of fault features is done on the score values. An opposition-based slime mould algorithm is investigated for finding the optimal parameters (weight connecting the input layer with output layer; and biases in hidden neurons) of ELM. Using optimized ELM parameters, the classification model is built. Test data evaluate the fitness of the built ELM model. Both the training and testing accuracy are 100%, with a computation time of 0.0023 s.
Faster R-CNN and YOLO based Vehicle detection: A Survey

Automatic moving vehicle detection plays a crucial and challenging role in performing intelligent traffic surveillance. Numerous research projects aiming to perform proper detection and tracking of vehicles have been carried out and the methods designed under these projects have found their uses in various important applications for e.g. to minimize the fatal accidents which mainly occur due to negligence of drivers or due to poor visibility during inclement weather condition or due to improper illumination, etc. At present, several deep neural networks have been proposed for performing object detection. This paper presents a comprehensive review of existing Faster Region-based Convolutional Neural Network (Faster R-CNN) and You look only once (YOLO) based vehicle detection and tracking methods. In this survey, we have divided the existing vehicle detection methods into different groups depending upon the architecture (Faster R-CNN/YOLO) which have been used as the backbone of these designed methods. We have organized the entire survey in chronological order so that interrelations between proposed methods can be highlighted. Apart from performing in depth analyses of the existing methods, we have described the respective architectures of Faster R-CNN, YOLO and their proposed variants in details in this survey for better understanding. We have concluded this paper by listing down the limitations of the existing works and unexplored aspects of this research topic. We have also thrown some light on the future scope of this research area.
Self-Knowledge Distillation with Progressive Refinement of Targets

The generalization capability of deep neural networks has been substantially improved by applying a wide spectrum of regularization methods, e.g., restricting function space, injecting randomness during training, augmenting data, etc. In this work, we propose a simple yet effective regularization method named progressive self-knowledge distillation (PS-KD), which progressively distills a model’s own knowledge to soften hard targets (i.e., one-hot vectors) during training. Hence, it can be interpreted within a framework of knowledge distillation as a student becomes a teacher itself. Specifically, targets are adjusted adaptively by combining the ground-truth and past predictions from the model itself. We show that PS-KD provides an effect of hard example mining by rescaling gradients according to difficulty in classifying examples. The proposed method is applicable to any supervised learning tasks with hard targets and can be easily combined with existing regularization methods to further enhance the generalization performance. Furthermore, it is confirmed that PS-KD achieves not only better accuracy, but also provides high quality of confidence estimates in terms of calibration as well as ordinal ranking. Extensive experimental results on three different tasks, image classification, object detection, and machine translation, demonstrate that our method consistently improves the performance of the state-of-the-art baselines. The code is available at https://github.com/lgcnsai/PS-KD-Pytorch.
UW-GAN: Single-Image Depth Estimation and Image Enhancement for Underwater Images

Due to the unavailability of large scale underwater depth image datasets and ill-posed problems, underwater single image depth prediction is a challenging task. An unambiguous depth prediction for single underwater image is an essential part of applications like underwater robotics, marine engineering, etc. This paper presents an end-to-end Underwater Generative Adversarial Network (UW-GAN) for depth estimation from an underwater single image. Initially, a coarse-level depth map is estimated using the Underwater Coarse-level Generative Network (UWC-Net). Then, a fine-level depth map is computed using the Underwater Fine-level Network (UWF-Net) which takes input as the concatenation of the estimated coarse-level depth map and the input image. The proposed UWF-Net comprises of spatial and channel-wise squeeze and excitation block for fine-level depth estimation. Also, we propose a synthetic underwater image generation approach for large scale database. The proposed network is tested on real-world and synthetic underwater datasets for its performance analysis. We also perform a complete evaluation of the proposed UW-GAN on underwater images having different color domination, contrast, and lighting conditions. Presented UW-GAN framework is also investigated for underwater single image enhancement. Extensive result analysis proves the superiority of proposed UW-GAN over the state-of-the-art hand-crafted, and learning based approaches for underwater single image depth estimation and enhancement.
Face Sketch Synthesis by Multidomain Adversarial Learning

Given a training set of face photo-sketch pairs, face sketch synthesis targets at learning a mapping from the photo domain to the sketch domain. Despite the exciting progresses made in the literature, it retains as an open problem to synthesize high-quality sketches against blurs and deformations. Recent advances in generative adversarial training provide a new insight into face sketch synthesis, from which perspective the existing synthesis pipelines can be fundamentally revisited. In this paper, we present a novel face sketch synthesis method by multidomain adversarial learning (termed MDAL), which overcomes the defects of blurs and deformations toward high-quality synthesis. The principle of our scheme relies on the concept of “interpretation through synthesis.” In particular, we first interpret face photographs in the photodomain and face sketches in the sketch domain by reconstructing themselves respectively via adversarial learning. We define the intermediate products in the reconstruction process as latent variables, which form a latent domain. Second, via adversarial learning, we make the distributions of latent variables being indistinguishable between the reconstruction process of the face photograph and that of the face sketch. Finally, given an input face photograph, the latent variable obtained by reconstructing this face photograph is applied for synthesizing the corresponding sketch. Quantitative comparisons to the state-of-the-art methods demonstrate the superiority of the proposed MDAL method.
Provably efficient RL with Rich Observations via Latent State Decoding

We study the exploration problem in episodic MDPs with rich observations generated from a small number of latent states. Under certain identifiability assumptions, we demonstrate how to estimate a mapping from the observations to latent states inductively through a sequence of regression and clustering steps—where previously decoded latent states provide labels for later regression problems—and use it to construct good exploration policies. We provide finite-sample guarantees on the quality of the learned state decoding function and exploration policies, and complement our theory with an empirical evaluation on a class of hard exploration problems. Our method exponentially improves over Q-learning with naive exploration, even when Q-learning has cheating access to latent states.
DSSLIC: Deep Semantic Segmentation-based Layered Image Compression

Deep learning has revolutionized many computer vision fields in the last few years, including learning-based image compression. In this paper, we propose a deep semantic segmentation-based layered image compression (DSSLIC) framework in which the segmentation map of the input image is obtained and encoded as the base layer of the bit-stream. A compact representation of the input image is also generated and encoded as the first enhancement layer. The segmentation map and the compact version of the image are then employed to obtain a coarse reconstruction of the image. The residual between the input and the coarse reconstruction is additionally encoded as another enhancement layer. Experimental results show that the proposed framework outperforms the H.265/HEVC-based BPG and other codecs in both PSNR and MS-SSIM metrics in RGB domain. Besides, since semantic map is included in the bit-stream, the proposed scheme can facilitate many other tasks such as image search and object-based adaptive image compression <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Hint-Based Training for Non-Autoregressive Machine Translation

Zhuohan Li, Zi Lin, Di He, Fei Tian, Tao Qin, Liwei Wang, Tie-Yan Liu. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Property-Constrained Dual Learning for Video Summarization

Video summarization is the technique to condense large-scale videos into summaries composed of key-frames or key-shots so that the viewers can browse the video content efficiently. Recently, supervised approaches have achieved great success by taking advantages of recurrent neural networks (RNNs). Most of them focus on generating summaries by maximizing the overlap between the generated summary and the ground truth. However, they neglect the most critical principle, i.e., whether the viewer can infer the original video content from the summary. As a result, existing approaches cannot preserve the summary quality well and usually demand large amounts of training data to reduce overfitting. In our view, video summarization has two tasks, i.e., generating summaries from videos and inferring the original content from summaries. Motivated by this, we propose a dual learning framework by integrating the summary generation (primal task) and video reconstruction (dual task) together, which targets to reward the summary generator under the assistance of the video reconstructor. Moreover, to provide more guidance to the summary generator, two property models are developed to measure the representativeness and diversity of the generated summary. Practically, experiments on four popular data sets (SumMe, TVsum, OVP, and YouTube) have demonstrated that our approach, with compact RNNs as the summary generator, using less training data, and even in the unsupervised setting, can get comparable performance with those supervised ones adopting more complex summary generators and trained on more annotated data.
A Fair Comparison of Graph Neural Networks for Graph Classification

The graph representation learning field has recently attracted the attention of a wide research community. Several Graph Neural Network models are being developed to tackle effective graph classification. However, experimental procedures often lack rigorousness and are hardly reproducible. Motivated by this, we provide an overview of common practices that should be avoided to fairly compare with the state of the art. To counter this troubling trend, we ran more than 47000 experiments in a controlled and uniform framework to re-evaluate five popular models across nine common benchmarks. Moreover, by comparing GNNs with structure-agnostic baselines we provide convincing evidence that, on some datasets, structural information has not been exploited yet. We believe that this work can contribute to the development of the graph learning field, by providing a much needed grounding for rigorous evaluations of graph classification models.
RACE-Net: A Recurrent Neural Network for Biomedical Image Segmentation

The level set based deformable models (LDM) are commonly used for medical image segmentation. However, they rely on a handcrafted curve evolution velocity that needs to be adapted for each segmentation task. The Convolutional Neural Networks (CNN) address this issue by learning robust features in a supervised end-to-end manner. However, CNNs employ millions of network parameters, which require a large amount of data during training to prevent over-fitting and increases the memory requirement and computation time during testing. Moreover, since CNNs pose segmentation as a region-based pixel labeling, they cannot explicitly model the high-level dependencies between the points on the object boundary to preserve its overall shape, smoothness or the regional homogeneity within and outside the boundary. We present a Recurrent Neural Network based solution called the RACE-net to address the above issues. RACE-net models a generalized LDM evolving under a constant and mean curvature velocity. At each time-step, the curve evolution velocities are approximated using a feed-forward architecture inspired by the multiscale image pyramid. RACE-net allows the curve evolution velocities to be learned in an end-to-end manner while minimizing the number of network parameters, computation time, and memory requirements. The RACE-net was validated on three different segmentation tasks: optic disc and cup in color fundus images, cell nuclei in histopathological images, and the left atrium in cardiac MRI volumes. Assessment on public datasets was seen to yield high Dice values between 0.87 and 0.97, which illustrates its utility as a generic, off-the-shelf architecture for biomedical segmentation.
Hyperspectral Image Denoising via Subspace-Based Nonlocal Low-Rank and Sparse Factorization

Hyperspectral images (HSIs) are unavoidably contaminated by different types of noise during data acquisition and transmission, e.g., Gaussian noise, impulse noise, stripes, and deadlines. A variety of mixed noise reduction approaches are developed for HSI, in which the subspace-based methods have achieved comparable performance. In this paper, a novel subspace-based nonlocal low-rank and sparse factorization (SNLRSF) method is proposed to remove the mixture of several types of noise. The SNLRSF method explores spectral low rank based on the fact that spectral signatures of pixels lie in a low-dimensional subspace and employs the nonlocal low-rank factorization to take the spatial nonlocal self-similarity into consideration. At the same time, the successive singular value decomposition (SVD) low-rank factorization algorithm is used to estimate three-dimensional (3-D) tensor generated by nonlocal similar 3-D patches. Moreover, the well-known augmented Lagrangian method is adopted to solve final denoising model efficiently. The experimental results over simulated and real datasets demonstrate that the proposed approach outperforms the related state-of-the-art methods in terms of visual quality and quantitative evaluation.