Reviewerly/exHarmony huggingface dataset:
Automatic estimation of parametric saliency maps (PSMs) for autonomous pedestrians

Modelling visual attention is an important aspect of simulating realistic virtual humans. This work proposes a parametric model and method for generating real-time saliency maps from the perspective of virtual agents which approximate those of vision-based saliency approaches. The model aggregates a saliency score from user-defined parameters for objects and characters in an agent’s view and uses that to output a 2D saliency map which can be modulated by an attention field to incorporate 3D information as well as a character’s state of attentiveness. The aggregate and parameterized structure of the method allows the user to model a range of diverse agents. The user may also expand the model with additional layers and parameters. The proposed method can be combined with normative and pathological models of the human visual field and gaze controllers, such as the recently proposed model of egocentric distractions for casual pedestrians that we use in our results. This is an extended version of a short paper published in MIG 2021. The extension includes an optimization approach that fits the parameters of the proposed model to established saliency models such as SALICON using a much larger and more realistic urban test set.
Sentiment Interaction Distillation Network for Image Sentiment Analysis

Sentiment is a high-level abstraction, and it is a challenging task to accurately extract sentimental features from visual contents due to the “affective gap”. Previous works focus on extracting more concrete sentimental features of individual objects by introducing saliency detection or instance segmentation into their models, neglecting the interaction among objects. Inspired by the observation that interaction among objects can impact the sentiment of images, we propose the Sentiment Interaction Distillation (SID) Network, which utilizes object sentimental interaction to guide feature learning. Specifically, we first utilize a panoptic segmentation method to obtain objects in images; then, we propose a sentiment-related edge generation method and employ Graph Convolution Network to aggregate and propagate object relation representation. In addition, we propose a knowledge distillation framework to utilize interaction information guiding global context feature learning, which can avoid noisy features introduced by error propagation and a varying number of objects. Experimental results show that our method outperforms the state-of-the-art algorithm, e.g., about 1.2% improvement on the Flickr dataset and 1.7% on the most challenging subset of Twitter I. It is demonstrated that the reasonable use of interaction features can improve the performance of sentiment analysis.
Global User-Level Perception of COVID-19 Contact Tracing Applications: Data-Driven Approach Using Natural Language Processing

Contact tracing has been globally adopted in the fight to control the infection rate of COVID-19. To this aim, several mobile apps have been developed. However, there are ever-growing concerns over the working mechanism and performance of these applications. The literature already provides some interesting exploratory studies on the community's response to the applications by analyzing information from different sources, such as news and users' reviews of the applications. However, to the best of our knowledge, there is no existing solution that automatically analyzes users' reviews and extracts the evoked sentiments. We believe such solutions combined with a user-friendly interface can be used as a rapid surveillance tool to monitor how effective an application is and to make immediate changes without going through an intense participatory design method.In this paper, we aim to analyze the efficacy of AI and NLP techniques for automatically extracting and classifying the polarity of users' sentiments by proposing a sentiment analysis framework to automatically analyze users' reviews on COVID-19 contact tracing mobile apps. We also aim to provide a large-scale annotated benchmark data set to facilitate future research in the domain. As a proof of concept, we also developed a web application based on the proposed solutions, which is expected to help the community quickly analyze the potential of an application in the domain.We propose a pipeline starting from manual annotation via a crowd-sourcing study and concluding with the development and training of artificial intelligence (AI) models for automatic sentiment analysis of users' reviews. In detail, we collected and annotated a large-scale data set of user reviews on COVID-19 contact tracing applications. We used both classical and deep learning methods for classification experiments.We used 8 different methods on 3 different tasks, achieving up to an average F1 score of 94.8%, indicating the feasibility of the proposed solution. The crowd-sourcing activity resulted in a large-scale benchmark data set composed of 34,534 manually annotated reviews.The existing literature mostly relies on the manual or exploratory analysis of users' reviews on applications, which is tedious and time-consuming. In existing studies, generally, data from fewer applications are analyzed. In this work, we showed that AI and natural language processing techniques provide good results for analyzing and classifying users' sentiments' polarity and that automatic sentiment analysis can help to analyze users' responses more accurately and quickly. We also provided a large-scale benchmark data set. We believe the presented analysis, data set, and proposed solutions combined with a user-friendly interface can be used as a rapid surveillance tool to analyze and monitor mobile apps deployed in emergency situations leading to rapid changes in the applications without going through an intense participatory design method.
A Survey: Image Classification Models Based on Convolutional Neural Networks

In recent years, image classification has become one of the hot spots in the field of artificial intelligence and computer vision. Image classification is a technique that simulates the human visual system to identify objects and classify images accurately, which has wide application in the fields of image segmentation, object tracking, face recognition, and etc. With the rapid development of deep learning architecture and hardware computing power, image classification methods based on convolutional neural networks have been rapidly developed. In this paper, we review architectures of convolutional neural networks, the process of image classification and put emphasis on the principles of advanced neural network models, quantization and pruning methods. We also analyze experiments based on different models and methods to compare their experimental data on different dimensions in different application situations. Additionally, we discuss some important issues and research dilemmas combined with current research progress. Finally, future research trends relating to classification methods are also provided.
Dynamic image clustering from projected coordinates of deep similarity learning

Commonly used clustering algorithms typically require user parameters such as the number of clusters to be divided. Density-based algorithms do not have such requirements but are not suitable for high dimensional data. Recent studies have merged the cluster assignment task with deep similarity learning. In this paper, we propose a novel framework to perform dynamic image clustering without prior knowledge of the cluster count. A deep learning model first learns data similarity from scratch, followed by the use of a coordinate learning model to project high dimensional data onto a two-dimensional space. A new clustering algorithm, raster clustering, is proposed to evaluate and classify the projected data. This mechanism can be applied in high dimensional data clustering like image data, and it allows the prediction of unseen data in a consistent way without the need for consolidating with training data.
K Nearest Neighbor OveRsampling approach: An open source python package for data augmentation

Data is present in abundance, but the problem of imbalanced dataset crops up time and again, vexing classifiers and reducing accuracy. This paper introduces K Nearest Neighbor OveRsampling (KNNOR) Algorithm — a novel data augmentation technique that considers the distribution of data and takes into account the k nearest neighbors while generating artificial data points. The KNNOR algorithm has outperformed the state-of-the-art augmentation algorithms by enabling classifiers to achieve much higher accuracy after injecting artificial minority datapoints into imbalanced datasets. This method is useful especially in health datasets where an imbalance is common and can even be applied to images of lower dimensions.
Multi-Modality Microscopy Image Style Augmentation for Nuclei Segmentation

Annotating microscopy images for nuclei segmentation by medical experts is laborious and time-consuming. To leverage the few existing annotations, also across multiple modalities, we propose a novel microscopy-style augmentation technique based on a generative adversarial network (GAN). Unlike other style transfer methods, it can not only deal with different cell assay types and lighting conditions, but also with different imaging modalities, such as bright-field and fluorescence microscopy. Using disentangled representations for content and style, we can preserve the structure of the original image while altering its style during augmentation. We evaluate our data augmentation on the 2018 Data Science Bowl dataset consisting of various cell assays, lighting conditions, and imaging modalities. With our style augmentation, the segmentation accuracy of the two top-ranked Mask R-CNN-based nuclei segmentation algorithms in the competition increases significantly. Thus, our augmentation technique renders the downstream task more robust to the test data heterogeneity and helps counteract class imbalance without resampling of minority classes.
Posture Correction and Detection using 3-D Image Classification

Posture detection is quite challenging as it involves a lot of intricacies for correctly identifying the key points involved. In this paper, we analyzed existing Posture detection models and figured out the shortcomings associated with them. Half of them involve hardware, making them immovable and inconvenient for the user. The other half engages with a recognition system based on sensors that involve high costs and complex implementation processes. Scrutinizing them one after another, we developed a comprehensive hybrid model that converts a 2-dimensional scene into a 3-dimensional one, making it easier &amp; quicker for someone sitting in front of a screen to correct their posture. This model can capture the temporal evolution of information offering insights into the user&#x0027;s posture at different times. It helps in achieving better results than the existing models.
Modeling and mapping of Inselberg habitats for environmental conservation in the Atlantic Forest and Caatinga domains, Brazil

Inselbergs are rocky outcrops generally composed of granitic or gneiss matrix, with monolithic proportions and expressive levels of biodiversity and endemism in different climatic domains. Brazil has one of the largest concentrations of Inselbergs on the planet, which require research to highlight the representative sites for conservation in the urban and rural environment. In this context, the present study evaluated the performance of machine learning algorithms in the mapping of Inselbergs in different environments. Specific objectives correspond to predicting and spatializing Inselbergs using algorithms, as well as selecting important covariates for their spatial distribution in the Atlantic climatic domain of Brazil. Thus, image classification techniques, machine learning algorithms and geospatial modeling were used to map the distribution of Inselbergs and select relevant sites for ecological and environmental conservation, considering anthropogenic and climate change threats. According to the results, the employed models presented kappa and accuracy values ranging from 0.76 to 0.91 and 0.89 to 0.96, respectively. Gradient Boosting Machine (GBM), Support Vector Machines with Radial Basis Function Kernel (SVM), C5.0 and Random Forest (RF) algorithms presented the best results in the mapping of the typical Inselbergs in the study area, since they were able to model spectral signatures of different land use and land cover classes. Statistically, they do not depend on the normal distribution of data and, at the same time, algorithms are able to work with a high number of covariates. The employed methodology has great potential to help in the selection of typical Inselberg landscapes in the Atlantic Forest and Caatinga biomes, which can support the protection and efficient management of biodiversity and geodiversity. The proposed methodology can be adapted to different areas and biomes of the world.
Single-Shot Intrinsic Calibration for Autonomous Driving Applications

In this paper, we present a first-of-its-kind method to determine clear and repeatable guidelines for single-shot camera intrinsic calibration using multiple checkerboards. With the help of a simulator, we found the position and rotation intervals that allow optimal corner detector performance. With these intervals defined, we generated thousands of multiple checkerboard poses and evaluated them using ground truth values, in order to obtain configurations that lead to accurate camera intrinsic parameters. We used these results to define guidelines to create multiple checkerboard setups. We tested and verified the robustness of the guidelines in the simulator, and additionally in the real world with cameras with different focal lengths and distortion profiles, which help generalize our findings. Finally, we used a 3D LiDAR (Light Detection and Ranging) to project and confirm the quality of the intrinsic parameters projection. We found it possible to obtain accurate intrinsic parameters for 3D applications, with at least seven checkerboard setups in a single image that follow our positioning guidelines.
Imbalanced Classification via Feature Dictionary-Based Minority Oversampling

Image classification research is one of the fields continuously studied in the computer vision domain, and several related studies have been actively conducted until recently.However, a limit exists regarding the prediction performance of real-world datasets due to the data imbalance problem between classes.Data augmentation through artificial sample generation for minority classes is one of the methods used to overcome this limitation.Among the various oversampling methods, we propose the feature dictionary-based generative model for the oversampling method.Feature dictionaries are built through the pretrained feature extractor, and the proposed generative model synthesizes artificial samples based on the dictionary.Class-to-class balanced training can be conducted by fine-tuning the classifier as additional data for the minority class.We experiment by applying the proposed framework to the fashion dataset, which has an extreme class imbalance.The experimental results demonstrate that the proposed model achieved the highest top-1 performance on various public fashion datasets.In addition, we analyze the number of samples in the dictionary and test the effectiveness of the elements that comprise the proposed model using various ablation studies.
Improvement to LiDAR-camera extrinsic calibration by using 3D–3D correspondences

In data fusion, calibration between 3D light detection and ranging (LiDAR) systems and cameras is one of the most fundamental tasks. We propose a calibration method to improve calibration accuracy by using a planar board. Most previous studies have attempted to find correspondences between 3D point clouds and 2D image pixels. In this study, a rigid transformation between LiDAR and camera was performed by exploiting the 3D–3D vertices and plane correspondences. First, the features, vertices and plane, are extracted from the camera and LiDAR, respectively. By compensating for the missing part of the 3D point cloud at the edges of the board, we improve the accuracy of the vertex estimation instead of only using the traditional RANSAC algorithm. Second, the rigid transformation is estimated by matching these 3D correspondences and refined by an ICP improvement. Finally, to evaluate the robustness of the proposed calibration method, we performed experiments and analyzed the calibration errors in both simulated data and real datasets.
A Learning Based Contrast Specific no Reference Image Quality Assessment Algorithm

Contrast is one of the most important visual characteristics of an image that has a significant effect in understanding an image, however, due to different imaging conditions and poor devices, quality of image in terms of contrast will degrade. although, limited methods have been used to assess the quality of a contrast distorted images. Proper image contrast enhancement can increase the perceptual quality of most contrast distorted images. In this paper, assuming that the output images of a contrast enhancing algorithms have a quality such as a reference image, a learning-based contrast-specific no reference image quality assessment method is proposed. In the proposed method in this paper the image with the closest quality to the reference image is selected using a pre-trained classification network, and then the quality assessment is performed by comparing the enhanced image and the distorted image using structural similarity (SSIM) index. The functionality of the proposed method has been validated using three well-known contrast distorted image datasets (CSIQ, CCID2014 and TID2013).
Reversible Data Hiding in Encrypted Images Based on Adaptive Prediction-error Label Map

In the field of reversible data hiding in encrypted images (RDH-EI), predict an image effectively and embed a message into the image with lower distortion are two crucial aspects.However, due to the linear regression prediction being sensitive to outliers, it is a challenge to improve the accuracy of predictions.To address this problem, this paper proposes an RDH-EI scheme based on adaptive prediction-error label map.In the prediction stage, an adaptive threshold estimation algorithm based on local complexity is proposed.Then, the pixels selection method based on gradient of image is designed to train the parameters of the prediction model.In the embedding stage, reserve enough space to embed auxiliary information and secret data embedding by flipping the least significant bits (LSBs) to encrypt the original image.In the receiver, based on the prediction-error map, error-free image recovery can be achieved.Extensive experimental results have shown that the proposed method can achieve effective pixel prediction results and get a higher embedding rate (ER).
Obstacles Detection for Electric Wheelchair with Computer Vision

This research aims to present the detection system of an obstacle for electric wheelchair using computer vision in order to facilitate for disabled persons and reduce the possibilities of accidents. In this system, the distance threshold is set to alert when a wheelchair is approaching an obstacle. The alert system consists of the smartphone's camera attached to the back of a wheelchair. The YOLOv3 model was used for object detection. The researcher has developed an algorithm to detect obstacles such as pillars, doors, or edge of the wall with edge detection method to enhance the detection efficiency of the system. Therefore, the usage of two algorithms enables the system to choose the obstacle detection between objects and edge detection. The research found that the system can choose the algorithm to detect obstacles with an accuracy of up to 80%. Moreover, the experiment revealed that the system can alert warnings before collisions with an accuracy of up to 90%. Further, this system can also calculate the approximate time prior to the collision.
Comparison Analysis of Traditional Machine Learning and Deep Learning Techniques for Data and Image Classification

The purpose of the study is to analyse and compare the most common machine learning and deep learning techniques used for computer vision 2D object classification tasks. Firstly, we will present the theoretical background of the Bag of Visual words model and Deep Convolutional Neural Networks (DCNN). Secondly, we will implement a Bag of Visual Words model, the VGG16 CNN Architecture. Thirdly, we will present our custom and novice DCNN in which we test the aforementioned implementations on a modified version of the Belgium Traffic Sign dataset. Our results showcase the effects of hyperparameters on traditional machine learning and the advantage in terms of accuracy of DCNNs compared to classical machine learning methods. As our tests indicate, our proposed solution can achieve similar - and in some cases better - results than existing DCNNs architectures. Finally, the technical merit of this article lies in the presented computationally simpler DCNN architecture, which we believe can pave the way towards using more efficient architectures for basic tasks.
Spider Taylor-ChOA: Optimized Deep Learning Based Sentiment Classification for Review Rating Prediction

The prediction of review rating is an imperative sentiment assessment task that aims to discover the intensity of users’ sentiment toward a target product from several reviews. This paper devises a technique based on sentiment classification for predicting the review rating. Here, the review data are taken from the database. The significant features, such as SentiWordNet-based statistical features, term frequency–inverse document frequency (TF-IDF), number of capitalized words, numerical words, punctuation marks, elongated words, hashtags, emoticons, and number of sentences are mined in feature extraction. The features are mined for sentiment classification, which is performed by random multimodal deep learning (RMDL). The training of RMDL is done using the proposed Spider Taylor-ChOA, which is devised by combining spider monkey optimization (SMO) and Taylor-based chimp optimization algorithm (Taylor-ChOA). Concurrently, the features are considered input for the review rating prediction, which determines positive and negative reviews using the hierarchical attention network (HAN), and training is done using proposed Spider Taylor-ChOA. The proposed Spider Taylor-ChOA-based RMDL performed best with the highest precision of 94.1%, recall of 96.5%, and highest F-measure of 95.3%. The proposed spider Taylor-ChOA-based HAN performed best with the highest precision of 93.1%, recall of 95.4% and highest F-measure of 94.3%.
What is That? Crowdsourcing Questions to a Virtual Exhibition

Virtual environments with an ambient natural interface that allows to retrieve information for learning about the environment are a promising combination for implementing engaging virtual exhibitions. As a step towards better understanding search behavior in such exhibitions, this paper contributes the data from an exploratory study with participants asking questions on a real-world historical room, the Gropiuszimmer at the Weimar Bauhaus, while being on an "online virtual tour" through the room. The dataset comprises 849 manually categorized questions (557 in English, 292 in German) from 63 participants combined with a detailed interaction log, which allows replaying each session (29 hours total). The presented dataset and analyses aim to provide researchers and practitioners with a starting point to develop in-depth studies and prototypical systems.