Reviewerly/exHarmony huggingface dataset:
Towards fairer datasets: filtering and balancing the distribution of the people subtree in the ImageNet hierarchy

Computer vision technology is being used by many but remains representative of only a few. People have reported misbehavior of computer vision models, including offensive prediction results and lower performance for underrepresented groups. Current computer vision models are typically developed using datasets consisting of manually annotated images or videos; the data and label distributions in these datasets are critical to the models' behavior. In this paper, we examine ImageNet, a large-scale ontology of images that has spurred the development of many modern computer vision methods. We consider three key factors within the person subtree of ImageNet that may lead to problematic behavior in downstream computer vision technology: (1) the stagnant concept vocabulary of WordNet, (2) the attempt at exhaustive illustration of all categories with images, and (3) the inequality of representation in the images within concepts. We seek to illuminate the root causes of these concerns and take the first steps to mitigate them constructively.
Unsupervised anomaly segmentation via deep feature reconstruction

Automatic detecting anomalous regions in images of objects or textures without priors of the anomalies is challenging, especially when the anomalies appear in very small areas of the images, making difficult-to-detect visual variations, such as defects on manufacturing products. This paper proposes an effective unsupervised anomaly segmentation approach that can detect and segment out the anomalies in small and confined regions of images. Concretely, we develop a multi-scale regional feature generator which can generate multiple spatial context-aware representations from pre-trained deep convolutional networks for every subregion of an image. The regional representations not only describe the local characteristics of corresponding regions but also encode their multiple spatial context information, making them discriminative and very beneficial for anomaly detection. Leveraging these descriptive regional features, we then design a deep yet efficient convolutional autoencoder and detect anomalous regions within images via fast feature reconstruction. Our method is simple yet effective and efficient. It advances the state-of-the-art performances on several benchmark datasets and shows great potential for real applications.
Sentiment Analysis of Lockdown in India During COVID-19: A Case Study on Twitter

With the rapid increase in the use of the Internet, sentiment analysis has become one of the most popular fields of natural language processing (NLP). Using sentiment analysis, the implied emotion in the text can be mined effectively for different occasions. People are using social media to receive and communicate different types of information on a massive scale during COVID-19 outburst. Mining such content to evaluate people's sentiments can play a critical role in making decisions to keep the situation under control. The objective of this study is to mine the sentiments of Indian citizens regarding the nationwide lockdown enforced by the Indian government to reduce the rate of spreading of Coronavirus. In this work, the sentiment analysis of tweets posted by Indian citizens has been performed using NLP and machine learning classifiers. From April 5, 2020 to April 17, 2020, a total of 12 741 tweets having the keywords "Indialockdown" are extracted. Data have been extracted from Twitter using Tweepy API, annotated using TextBlob and VADER lexicons, and preprocessed using the natural language tool kit provided by the Python. Eight different classifiers have been used to classify the data. The experiment achieved the highest accuracy of 84.4% with LinearSVC classifier and unigrams. This study concludes that the majority of Indian citizens are supporting the decision of the lockdown implemented by the Indian government during corona outburst.
Bidirectional Machine Reading Comprehension for Aspect Sentiment Triplet Extraction

Aspect sentiment triplet extraction (ASTE), which aims to identify aspects from review sentences along with their corresponding opinion expressions and sentiments, is an emerging task in fine-grained opinion mining. Since ASTE consists of multiple subtasks, including opinion entity extraction, relation detection, and sentiment classification, it is critical and challenging to appropriately capture and utilize the associations among them. In this paper, we transform ASTE task into a multi-turn machine reading comprehension (MTMRC) task and propose a bidirectional MRC (BMRC) framework to address this challenge. Specifically, we devise three types of queries, including non-restrictive extraction queries, restrictive extraction queries and sentiment classification queries, to build the associations among different subtasks. Furthermore, considering that an aspect sentiment triplet can derive from either an aspect or an opinion expression, we design a bidirectional MRC structure. One direction sequentially recognizes aspects, opinion expressions, and sentiments to obtain triplets, while the other direction identifies opinion expressions first, then aspects, and at last sentiments. By making the two directions complement each other, our framework can identify triplets more comprehensively. To verify the effectiveness of our approach, we conduct extensive experiments on four benchmark datasets. The experimental results demonstrate that BMRC achieves state-of-the-art performances.
Improving data hiding within colour images using hue component of HSV colour space

Data hiding technologies aim to hide the existence of secret information within digital covers such as images by causing unnoticeable degradation to their quality. Reducing the image distortion and increasing the embedding capacity are the main points that the data hiding techniques revolved around. This article proposes two high payload embedding methods with high stego image quality using the Hue-Saturation-Value (HSV) colour model. The first method is hue-based embedding (HBE) that employs the H plane for hiding one or two bits in non-grey pixels. The second method uses the three HSV components, so it is called three-planes embedding (TPE). In TPE, one bit is hidden in the least significant bit (LSB) of V of the grey pixels, one or two bits in H of the pixels having low saturation or low brightness and one bit in the LSB of S otherwise. The experiments were conducted on 25 images and the results show that HBE hides more data on average than TPE with its quality reaching 60 dB. TPE achieves quality up to 61 dB and capacity reaches 364 Kb. TPE scores the highest capacity among six state-of-the-art techniques in Red-Green-Blue, HSV, Hue-Saturation-Intensity and YCbCr spaces with the highest average peak signal to noise ratio midst five of them. By embedding 60, 90, and 120 Kb, this TPE attains the best average quality amid all the methods.
Light GBM Machine Learning Algorithm to Online Click Fraud Detection

In the current web advertising activities, the fraud increases the number of risks for online marketing, advertising industry and e-business. The click fraud is considered one of the most critical issues in online advertising. Even if the online advertisers make permanent efforts to improve the traffic filtering techniques, they are still looking for the best protection methods to detect click frauds.
Ultracompact meta-imagers for arbitrary all-optical convolution

Electronic digital convolutions could extract key features of objects for data processing and information identification in artificial intelligence, but they are time-cost and energy consumption due to the low response of electrons. Although massless photons enable high-speed and low-loss analog convolutions, two existing all-optical approaches including Fourier filtering and Green's function have either limited functionality or bulky volume, thus restricting their applications in smart systems. Here, we report all-optical convolutional computing with a metasurface-singlet or -doublet imager, considered as the third approach, where its point spread function is modified arbitrarily via a complex-amplitude meta-modulator that enables functionality-unlimited kernels. Beyond one- and two-dimensional spatial differentiation, we demonstrate real-time, parallel, and analog convolutional processing of optical and biological specimens with challenging pepper-salt denoising and edge enhancement, which significantly enrich the toolkit of all-optical computing. Such meta-imager approach bridges multi-functionality and high-integration in all-optical convolutions, meanwhile possessing good architecture compatibility with digital convolutional neural networks.
Survey on Synthetic Data Generation, Evaluation Methods and GANs

Synthetic data consists of artificially generated data. When data are scarce, or of poor quality, synthetic data can be used, for example, to improve the performance of machine learning models. Generative adversarial networks (GANs) are a state-of-the-art deep generative models that can generate novel synthetic samples that follow the underlying data distribution of the original dataset. Reviews on synthetic data generation and on GANs have already been written. However, none in the relevant literature, to the best of our knowledge, has explicitly combined these two topics. This survey aims to fill this gap and provide useful material to new researchers in this field. That is, we aim to provide a survey that combines synthetic data generation and GANs, and that can act as a good and strong starting point for new researchers in the field, so that they have a general overview of the key contributions and useful references. We have conducted a review of the state-of-the-art by querying four major databases: Web of Sciences (WoS), Scopus, IEEE Xplore, and ACM Digital Library. This allowed us to gain insights into the most relevant authors, the most relevant scientific journals in the area, the most cited papers, the most significant research areas, the most important institutions, and the most relevant GAN architectures. GANs were thoroughly reviewed, as well as their most common training problems, their most important breakthroughs, and a focus on GAN architectures for tabular data. Further, the main algorithms for generating synthetic data, their applications and our thoughts on these methods are also expressed. Finally, we reviewed the main techniques for evaluating the quality of synthetic data (especially tabular data) and provided a schematic overview of the information presented in this paper.
An Approximate Schur Decomposition-Based Spatial Domain Color Image Watermarking Method

In this paper, an approximate Schur decomposition-based spatial domain blind color image watermarking method is proposed to protect the copyright of color images, which has low computation complexity similiar to the watermarking technique in the spatial domain and strong robustness similiar to the watermarking technique in the transform domain. First, the approximate maximum eigenvalue of Schur decomposition is calculated in the spatial domain by the proposed method. Second, the approximate maximum eigenvalue is used to embed and extract the color watermark image in the spatial domain without the true Schur decomposition. Moreover, the procedures of the proposed watermarking method are given in detail. The proposed technique is performed on the spatial domain based on the approximate Schur decomposition and belongs to blind watermarking technique. The experimental results on two publicly available image databases (CVG-UGR and USC-SIPI) have demonstrated the effectiveness of the proposed method in terms of invisibility, robustness, and the real-time feature.
Combined Reinforcement Learning via Abstract Representations

In the quest for efficient and robust reinforcement learning methods, both model-free and model-based approaches offer advantages. In this paper we propose a new way of explicitly bridging both approaches via a shared low-dimensional learned encoding of the environment, meant to capture summarizing abstractions. We show that the modularity brought by this approach leads to good generalization while being computationally efficient, with planning happening in a smaller latent state space. In addition, this approach recovers a sufficient low-dimensional representation of the environment, which opens up new strategies for interpretable AI, exploration and transfer learning.
Staining Invariant Features for Improving Generalization of Deep Convolutional Neural Networks in Computational Pathology

One of the main obstacles for the implementation of deep convolutional neural networks (DCNNs) in the clinical pathology workflow is their low capability to overcome variability in slide preparation and scanner configuration, that leads to changes in tissue appearance. Some of these variations may not be not included in the training data, which means that the models have a risk to not generalize well. Addressing such variations and evaluating them in reproducible scenarios allows understanding of when the models generalize better, which is crucial for performance improvements and better DCNN models. Staining normalization techniques (often based on color deconvolution and deep learning) and color augmentation approaches have shown improvements in the generalization of the classification tasks for several tissue types. Domain-invariant training of DCNN's is also a promising technique to address the problem of training a single model for different domains, since it includes the source domain information to guide the training toward domain-invariant features, achieving state-of-the-art results in classification tasks. In this article, deep domain adaptation in convolutional networks (DANN) is applied to computational pathology and compared with widely used staining normalization and color augmentation methods in two challenging classification tasks. The classification tasks rely on two openly accessible datasets, targeting Gleason grading in prostate cancer, and mitosis classification in breast tissue. The benchmark of the different techniques and their combination in two DCNN architectures allows us to assess the generalization abilities and advantages of each method in the considered classification tasks. The code for reproducing our experiments and preprocessing the data is publicly available. Quantitative and qualitative results show that the use of DANN helps model generalization to external datasets. The combination of several techniques to manage color heterogeneity suggests that several methods together, such as color augmentation methods with DANN training, can generalize even further. The results do not show a single best technique among the considered methods, even when combining them. However, color augmentation and DANN training obtain most often the best results (alone or combined with color normalization and color augmentation). The statistical significance of the results and the embeddings visualizations provide useful insights to design DCNN that generalizes to unseen staining appearances. Furthermore, in this work, we release for the first time code for DANN evaluation in open access datasets for computational pathology. This work opens the possibility for further research on using DANN models together with techniques that can overcome the tissue preparation differences across datasets to tackle limited generalization.
Robust Navigation with Language Pretraining and Stochastic Sampling

Core to the vision-and-language navigation (VLN) challenge is building robust instruction representations and action decoding schemes, which can generalize well to previously unseen instructions and environments.In this paper, we report two simple but highly effective methods to address these challenges and lead to a new state-of-the-art performance.First, we adapt large-scale pretrained language models to learn text representations that generalize better to previously unseen instructions.Second, we propose a stochastic sampling scheme to reduce the considerable gap between the expert actions in training and sampled actions in test, so that the agent can learn to correct its own mistakes during long sequential action decoding.Combining the two techniques, we achieve a new state of the art on the Room-to-Room benchmark with 6% absolute gain over the previous best result (47% → 53%) on the Success Rate weighted by Path Length metric.2 To compensate, beam search is often used to improve success rates.Recent work, e.g., using search strategies (Ke et al., 2019) or progress monitors (Ma et al., 2019b), has focused on mitigating the cost of computing top-k rollouts.3 PRETRAINED LMS AND STOCHASTIC SAMPLING
Neural Tangents: Fast and Easy Infinite Neural Networks in Python

Neural Tangents is a library for working with infinite-width neural networks. It provides a high-level API for specifying complex and hierarchical neural network architectures. These networks can then be trained and evaluated either at finite-width as usual, or in their infinite-width limit. For the infinite-width networks, Neural Tangents performs exact inference either via Bayes' rule or gradient descent, and generates the corresponding Neural Network Gaussian Process and Neural Tangent kernels. Additionally, Neural Tangents provides tools to study gradient descent training dynamics of wide but finite networks. The entire library runs out-of-the-box on CPU, GPU, or TPU. All computations can be automatically distributed over multiple accelerators with near-linear scaling in the number of devices. In addition to the repository below, we provide an accompanying interactive Colab notebook at https://colab.sandbox.google.com/github/neural-tangents/neural-tangents/blob/master/notebooks/neural_tangents_cookbook.ipynb
CNN-Based Person Detection Using Infrared Images for Night-Time Intrusion Warning Systems

Night-time surveillance is important for safety and security purposes. For this reason, several studies have attempted to automatically detect people intruding into restricted areas by using infrared cameras. However, detecting people from infrared CCTV (closed-circuit television) is challenging because they are usually installed in overhead locations and people only occupy small regions in the resulting image. Therefore, this study proposes an accurate and efficient method for detecting people in infrared CCTV images during the night-time. For this purpose, three different infrared image datasets were constructed; two obtained from an infrared CCTV installed on a public beach and another obtained from a forward looking infrared (FLIR) camera installed on a pedestrian bridge. Moreover, a convolution neural network (CNN)-based pixel-wise classifier for fine-grained person detection was implemented. The detection performance of the proposed method was compared against five conventional detection methods. The results demonstrate that the proposed CNN-based human detection approach outperforms conventional detection approaches in all datasets. Especially, the proposed method maintained F1 scores of above 80% in object-level detection for all datasets. By improving the performance of human detection from infrared images, we expect that this research will contribute to the safety and security of public areas during night-time.
Unsupervised feature selection based extreme learning machine for clustering

For data with various complicated distribution in the original feature space, it is difficult to find the clusters of the data. Extreme learning machine (ELM) is famous for its universal approximation capability and the hidden space created by random nonlinear feature mapping. Existing ELM based clustering methods address this by constructing an embedding space, in which the cluster are easily revealed. A commonality of them is the final results have to be subsequently derived by k-means clustering. In this paper, we propose an unsupervised feature selection based extreme learning machine (UFSELM) for clustering, which integrates ELM with L2,1 norm regularization to remove the worthless hidden neurons and clusters the data directly without building an embedding. Specifically, the proposed method conducts feature selection by minimizing the L2,1 norm of output weights, and the clustering results is computed by eigendecomposition. By solving the formulated optimization problem in an iterative fashion, we improved the accuracy of clustering. We conducted experiments on several public datasets to demonstrate the effectiveness of the proposed method and further analyzed the properties of the proposed method.
MDSSD: multi-scale deconvolutional single shot detector for small objects

For most of the object detectors based on multi-scale feature maps, the shallow layers are rich in fine spatial information and thus mainly responsible for small object detection. The performance of small object detection, however, is still less than satisfactory because of the deficiency of semantic information on shallow feature maps. In this paper, we design a Multi-scale Deconvolutional Single Shot Detector (MDSSD), especially for small object detection. In MDSSD, multiple high-level feature maps at different scales are upsampled simultaneously to increase the spatial resolution. Afterwards, we implement the skip connections with low-level feature maps via Fusion Block. The fusion feature maps, named Fusion Module, are of strong feature representational power of small instances. It is noteworthy that these high-level feature maps utilized in Fusion Block preserve both strong semantic information and some fine details of small instances, rather than the top-most layer where the representation of fine details for small objects are potentially wiped out. The proposed framework achieves 77.6% mAP for small object detection on the challenging dataset TT100K with 512 x 512 input, outperforming other detectors with a large margin. Moreover, it can also achieve state-of-the-art results for general object detection on PASCAL VOC2007 test and MS COCO test-dev2015, especially achieving 2 to 5 points improvement on small object categories.
Wavelength-independent optical fully differential operation based on the spin–orbit interaction of light

Optical technology may provide important architectures for future computing, such as analog optical computing and image processing. Compared with traditional electric operation, optical operation has shown some unique advantages including faster operating speeds and lower power consumption. Here, we propose an optical full differentiator based on the spin–orbit interaction of light at a simple optical interface. The optical differential operation is independent of the wavelength due to the purely geometric nature of the phenomenon. As an important application of the fully differential operation, the wavelength-independent image processing of edge detection is demonstrated. By adjusting the polarization of the incident beam, the one-dimension edge imaging at any desirable direction can be obtained. The wavelength-independent image processing of edge detection provides possible applications in autonomous driving, target recognition, microscopic imaging, and augmented reality.