Reviewerly/exHarmony huggingface dataset:
Adversarial Learning with Mask Reconstruction for Text-Guided Image Inpainting

Text-guided image inpainting aims to complete the corrupted patches coherent with both visual and textual context. On one hand, existing works focus on surrounding pixels of the corrupted patches without considering the objects in the image, resulting in the characteristics of objects described in text being painted on non-object regions. On the other hand, the redundant information in text may distract the generation of objects of interest in the restored image. In this paper, we propose an adversarial learning framework with mask reconstruction (ALMR) for image inpainting with textual guidance, which consists of a two-stage generator and dual discriminators. The two-stage generator aims to restore coarse-grained and fine-grained images, respectively. In particular, we devise a dual-attention module (DAM) to incorporate the word-level and sentence-level textual features as guidance on generating the coarse-grained and fine-grained details in the two stages. Furthermore, we design a mask reconstruction module (MRM) to penalize the restoration of the objects of interest with the given textual descriptions about the objects. For adversarial training, we exploit global and local discriminators for the whole image and corrupted patches, respectively. Extensive experiments conducted on CUB-200-2011, Oxford-102 and CelebA-HQ show the outperformance of the proposed ALMR (e.g., FID value is reduced from 29.69 to 14.69 compared with the state-of-the-art approach on CUB-200-2011). Codes are available at https://github.com/GaranWu/ALMR
GLM-Net

In this work, we study the problem of separating the global camera motion and the local dynamic motion from an optical flow. Previous methods either estimate global motions by a parametric model, such as a homography, or estimate both of them by an optical flow field. However, none of these methods can directly estimate global and local motions through an end-to-end manner. In addition, separating the two motions accurately from a hybrid flow field is challenging. Because one motion can easily confuse the estimate of the other one when they are compounded together. To this end, we propose an end-to-end global and local motion estimation network GLM-Net. We design two encoder-decoder structures for the motion separation in the optical flow based on different task orientations. One structure adopts a mask autoencoder to extract the global motion, while the other one uses attention U-net for the local motion refinement. We further designed two effective training methods to overcome the problem of lacking supervisions. We apply our method on the action recognition datasets NCAA and UCF-101 to verify the accuracy of the local motion, and the homography estimation dataset DHE for the accuracy of the global motion. Experimental results show that our method can achieve competitive performance in both tasks at the same time, validating the effectiveness of the motion separation.
Local Integral Regression Network for Cell Nuclei Detection

Nuclei detection is a fundamental task in the field of histopathology image analysis and remains challenging due to cellular heterogeneity. Recent studies explore convolutional neural networks to either isolate them with sophisticated boundaries (segmentation-based methods) or locate the centroids of the nuclei (counting-based approaches). Although these two methods have demonstrated superior success, their fully supervised training demands considerable and laborious pixel-wise annotations manually labeled by pathology experts. To alleviate such tedious effort and reduce the annotation cost, we propose a novel local integral regression network (LIRNet) that allows both fully and weakly supervised learning (FSL/WSL) frameworks for nuclei detection. Furthermore, the LIRNet can output an exquisite density map of nuclei, in which the localization of each nucleus is barely affected by the post-processing algorithms. The quantitative experimental results demonstrate that the FSL version of the LIRNet achieves a state-of-the-art performance compared to other counterparts. In addition, the WSL version has exhibited a competitive detection performance and an effortless data annotation that requires only 17.5% of the annotation effort.
A Novel Big Data Storage Reduction Model for Drill Down Search

Multi-level searching is called Drill down search. Right now, no drill down search feature is available in the existing search engines like Google, Yahoo, Bing and Baidu. Drill down search is very much useful for the end user to find the exact search results among the huge paginated search results. Higher level of drill down search with category based search feature leads to get the most accurate search results but it increases the number and size of the file system. The purpose of this manuscript is to implement a big data storage reduction binary file system model for category based drill down search engine that offers fast multi-level filtering capability. The basic methodology of the proposed model stores the search engine data in the binary file system model. To verify the effectiveness of the proposed file system model, 5 million unique keyword data are stored into a binary file, thereby analysing the proposed file system with efficiency. Some experimental results are also provided based on real data that show our storage model speed and superiority. Experiments demonstrated that our file system expansion ratio is constant and it reduces the disk storage space up to 30% with conventional database/file system and it also increases the search performance for any levels of search. To discuss deeply, the paper starts with the short introduction of drill down search followed by the discussion of important technologies used to implement big data storage reduction system in detail.
Thermal Image Processing and Analysis for Surveillance UAVs

Forest fires solely are responsible for a multitude of environmental disasters. This in turn results in economic and ecological/environmental damage, thus bringing peril to people’s lives. Automatic surveillance vehicles and early detection in forest fires have gained more importance in comparison to the traditional human monitoring. This is because human monitoring is subjective, which affects reliability in detection and is one of the major concerns in forest-fire detection systems. In the present system, the process is time-consuming and false alarms must be declared manually. This research is an attempt to solve the aforementioned problem. The hypothesis is to deploy a drone equipped with a thermal camera to capture the area on fire and locate victims and give the shortest distance to them. This process is entirely done when the firefighters are in transit to the location. The main focus of this text is thermal video processing since it plays a crucial role in the entire prototype. This paper highlights the difference between two human detection algorithms: Histogram of Oriented Gradients (HOG) [1] and You Only Look Once (YOLOv2) [2, 3] for thermal videos. Furthermore, an algorithm for detecting fire in a thermal video has also been implemented. The inputs are various thermal videos which are available publicly. The conclusion is that the HOG algorithm is faster than the YOLO algorithm for thermal inputs but the latter is more accurate.
Parglare: A LR/GLR parser for Python

Parglare is a Python parsing library that implements deterministic LR and its generalized extension GLR algorithms. Parglare strives to be easy to use by providing rich error messages, visualization, a CLI tool for grammar development, and good documentation. The same grammar format is used for both algorithms. It is easy to choose either LR parsing if performance is more important or GLR in case a grammar cannot fit into the constraints of deterministic LR parsing and a more powerful parsing is needed. Parglare has been used in data extraction from various textual formats, analysis of legacy source code, and developing and teaching DSL development. The GitHub repository of the project contains multiple examples and a comprehensive functionality and performance test suite.
Transfer Evidential C-Means Clustering

Clustering is widely used in text analysis, natural language processing, image segmentation and other data mining fields. ECM (evidential c-means) is a powerful clustering algorithm developed in the theoretical framework of belief functions. Based on the concept of credal partition, it extends those of hard, fuzzy, and possibilistic clustering algorithms. However, as a clustering algorithm, it can only work well when the data is sufficient and the quality of the data is good. If the data is insufficient and the distribution is complex, or the data is sufficient but polluted, the clustering result will be poor. In order to solve this problem, using the strategy of transfer learning, this paper proposes a transfer evidential c-means (TECM) algorithm. TECM employs the historical clustering centers in source domain as the reference to guide the clustering in target domain. In addition, the proposed transfer clustering algorithm can adapt to situations where the number of clusters in source domain and target domain is different. The proposed algorithm has been validated on synthetic and real-world datasets. Experimental results demonstrate the effectiveness of transfer learning in comparison with ECM and the advantage of credal partition in comparison with TFCM.
Does the Layout Really Matter? A Study on Visual Model Accuracy Estimation

In visual interactive labeling, users iteratively assign labels to data items until the machine model reaches an acceptable accuracy. A crucial step of this process is to inspect the model’s accuracy and decide whether it is necessary to label additional elements. In scenarios with no or very little labeled data, visual inspection of the predictions is required. Similarity-preserving scatterplots created through a dimensionality reduction algorithm are a common visualization that is used in these cases. Previous studies investigated the effects of layout and image complexity on tasks like labeling. However, model evaluation has not been studied systematically. We present the results of an experiment studying the influence of image complexity and visual grouping of images on model accuracy estimation. We found that users outperform traditional automated approaches when estimating a model’s accuracy. Furthermore, while the complexity of images impacts the overall performance, the layout of the items in the plot has little to no effect on estimations.
Identifying characteristics of software vulnerabilities by their textual description using machine learning

Significant problems in the development of secure software are the complexity of formalizing software vulnerabilities and the lack of information security experts. The paper tests the hypothesis that it is possible to automate the process of identifying vulnerability characteristics. The method based on a proprietary scheme for classifying vulnerabilities and specialized for the Russia Government Database of Vulnerabilities is presented. The method classifies vulnerability characteristics (hazard level field) based on a human-centered description of the vulnerability (name field and details field) using a multiclass SVM model. Utility implementing the method was developed. A number of experiments were carried out using the utility: the calculation of the method quality, a random classification mode, and a multi-pass approach to collect statistical data on the quality of work. The paper substantiates the confirmation of the hypothesis of scientific research. The scheme of the method is described in an analytical form, the main algorithm of the utility operation and the experiment logs are given.
Hypergame-based Adaptive Behavior Path Planning for Combined Exploration and Visual Search

In this work, we present an adaptive behavior path planning method for autonomous exploration and visual search of unknown environments. As volumetric exploration and visual coverage of unknown environments, with possibly different sensors, are non-identical objectives, a principled combination of the two is proposed. In particular, the method involves three distinct planning policies, namely exploration, and sparse or dense visual coverage. A hypergame formulation is proposed which allows the robot to select for the next-best planning behavior in response to the currently encountered environment challenges in terms of geometry and visual conditions, alongside a self-assessment of its performance. The proposed planner is evaluated in a collection of experimental and simulation studies in diverse environments, while comparative results against a state-of-the-art exploration method are also presented.
Multi-Agent Deep Reinforcement Learning for Online 3D Human Poses Estimation

Most multi-view based human pose estimation techniques assume the cameras are fixed. While in dynamic scenes, the cameras should be able to move and seek the best views to avoid occlusions and extract 3D information of the target collaboratively. In this paper, we address the problem of online view selection for a fixed number of cameras to estimate multi-person 3D poses actively. The proposed method exploits a distributed multi-agent based deep reinforcement learning framework, where each camera is modeled as an agent, to optimize the action of all the cameras. An inter-agent communication protocol was developed to transfer the cameras’ relative positions between agents for better collaboration. Experiments on the Panoptic dataset show that our method outperforms other view selection methods by a large margin given an identical number of cameras. To the best of our knowledge, our method is the first to address online active multi-view 3D pose estimation with multi-agent reinforcement learning.
Novel Heuristic Scheme to Enforce Safety and Confidentiality Using Feature-Based Encryption in Multi-cloud Environment (MCE)

Safety is the main concern and also it acts as a major hurdle in maintaining cloud-based services. To overcome the difficulties, there exists a wide range of activities to be performed and a set of techniques that are carried out in cloud-based services. Apart from these safety issues, the cloud holds a collection of different features and frameworks. The cloud analyzes the security measures that are to be carried out to ensure its safety in providing services by developing diversified cloud services frequently. Such types of cloud frameworks are set off, and their safety, confidentiality, and probabilities are discussed in this chapter. To present twofold encryption and decryption to the users, the multi-cloud framework was built using two-stage encryption and decryption. The users of the cloud services should enjoy the salient features of the cloud through feature-based encryption standards. The main focus on cloud errors which in turn ends up in the loss of information from the cloud is taken into consideration while providing security in servers. The proposed technique offers a reduction of 0.3% and 5.62% system time for creating a directory for 10k and 20k files, respectively, when compared to the conventional B-Tree technique. The proposed technique of encryption provides twofold security and confidentiality of data to the end-users through feature-based encryption.
BERT-Based Meta-Learning Approach with Looking Back for Sentiment Analysis of Literary Book Reviews

Sentiment analysis of literary book reviews is crucial for authors to understand the voice of readers and for humanities researchers to investigate the reader reception of the literary book. Constructing an automated sentiment analysis system can help us analyze literary reviews in a big-data way, achieving a complete view of the full understanding. The main challenge in developing such a system is the lack of large-scale datasets with precise annotations. Some related researches collect the star rates from social networks and convert them into corresponding sentiment annotations for data labeling. Such automatically generated annotations can greatly reduce the human labeling efforts but with two defects. The first is that such annotations are filling with a vast amount of noise. For example, different people have significantly different perceptions for sentiment markers (positive, neutral, and negative) of a five-star comment system. Besides, literary book reviews contain more positive emotions than neutral and negative. Namely, this task is also challenged by imbalanced data. This paper introduces an automatically generated sentiment analysis dataset for Chinese literary book reviews with parts of manual verification, containing 187 literary books with 109,286 reviews. Furthermore, we propose a novel meta-learning approach with Looking Back mechanism to build a robust BERT-based sentiment classification model. Specifically, we design an extra meta-model for learning a sample weighting function from the historical sample information, which mitigates the influence of noisy labels and class imbalance problems. The parameters of this meta-model are finely updated with the whole training process by the guidance of a small amount of unbiased meta data. The results show the F1 score raise of 13.51% with the Looking Back mechanism, showing that our method can significantly promote sentiment analysis performance with noisy labels and class imbalance data.
What's Fair is Fair: Detecting and Mitigating Encoded Bias in Multimodal Models of Museum Visitor Attention

Recent years have seen growing interest in modeling visitor engagement in museums with multimodal learning analytics. In parallel, there has also been growing concern about issues of fairness and encoded bias in machine learning models. In this paper, we investigate bias detection and mitigation techniques to address issues of algorithmic fairness in multimodal models of museum visitor visual attention. We employ slicing analysis using the Absolute Between-ROC Area (ABROCA) statistic to detect encoded bias present in multimodal models of visitor visual attention trained with facial expression and posture data from visitor interactions with a game-based museum exhibit about environmental sustainability. We investigate instances of gender bias that arise between different combinations of modalities across several machine learning techniques. We also measure the effectiveness of two different debiasing strategies—learned fair representations and reweighing—when applied to the trained multimodal visitor attention models. Results indicate that patterns of bias can arise across different modality combinations for the different visitor visual attention models, and there is often an inherent tradeoff between predictive accuracy and ABROCA. Analyses suggest that debiasing strategies tend to be more effective on multimodal models of visitor visual attention than their unimodal counterparts
Oriented Spatial Correlative Aligned Feature for Remote Sensing Object Detection

In the past decade, the object detection in remote sensing has become a research focus with the development of aerial images' acquirement technology. However, it is difficult to accurately describe rotated rigid objects with large aspect ratio through axis-aligned convolutional features. To solve this problem, we propose an oriented spatial correlative feature alignment Net (OSCFA-Net) with a correlative feature alignment module (CFAM). CFAM aligns features by rotating object features and considering the spatial correlation of rigid objects to ensure the feature alignment between the internal feature points. Experiment results demonstrate that OSCFA-Net can achieve state-of-the-art performance in remote sensing object detection.
Glocal Alignment for Unsupervised Domain Adaptation

Traditional unsupervised domain adaptation methods attempt to align source and target domains globally and are agnostic to the categories of the data points. This results in an inaccurate categorical alignment and diminishes the classification performance on the target domain. In this paper, we alter existing adversarial domain alignment methods to adhere to category alignment by imputing category information. We partition the samples based on category using source labels and target pseudo labels and then apply domain alignment for every category. Our proposed modification provides a boost in performance even with a modest pseudo label estimator. We evaluate our approach on 4 popular domain alignment loss functions using object recognition and digit datasets.
Examining the Design and Development of a Social Justice Makerspace

In our society, where technologies continue to act as a marginalizing force for various individuals and communities, makerspaces present unique opportunities for community-driven work that engages people with a variety of communal technologies, knowledge, and work practices. However, makerspaces grew out of largely exclusionary discourses, which continue to neglect non-dominant technology practices, knowledge bases, and communities. Despite these origins, research remains largely optimistic about the potential for makerspaces to provide equitable opportunities for those stereotypically excluded to engage, learn, and build with technologies. We mirror this optimism through our work examining how makerspaces might be positioned to address social inequities while supporting community growth and development. This paper traces the development of a social justice makerspace situated within a community of color in the United States. The work contributes a contextual analysis of the design of a makerspace explicitly positioned to forefront community voices and address issues of social inequities. We highlight community perspectives within our situated design process as we draw out the themes and tensions that arose. Applying a social justice framework from Dombrowski et al. (2016) and Lötter's (2011) to our findings we demonstrate how makerspaces can be positioned to address social inequities through: identifying, educating, and communicating about inequities; engaging in the distribution of resources and opportunities; and enabling capacity building at the individual and structural levels. We highlight how an asset-based orientation can position the makerspace to grow in effective ways to address community issues.
Differential radial basis function network for sequence modelling

We propose a differential radial basis function (RBF) network termed RBF-DiffNet—whose hidden layer blocks are partial differential equations (PDEs) linear in terms of the RBF—to make the baseline RBF network robust to noise in sequential data. Assuming that the sequential data derives from the discretisation of the solution to an underlying PDE, the differential RBF network learns constant linear coefficients of the PDE, consequently regularising the RBF network by following modified backward-Euler updates. We experimentally validate the differential RBF network on the logistic map chaotic timeseries as well as on 30 real-world timeseries provided by Walmart in the M5 forecasting competition. The proposed model is compared with the normalised and unnormalised RBF networks, ARIMA, and ensembles of multilayer perceptrons (MLPs) and recurrent networks with long short-term memory (LSTM) blocks. From the experimental results, RBF-DiffNet consistently shows a marked reduction in the prediction error over the baseline RBF network (e.g., 41% reduction in the root mean squared scaled error on the M5 dataset, and 53% reduction in the mean absolute error on the logistic map); RBF-DiffNet also shows a comparable performance to the LSTM ensemble but requires 99% less computational time. Our proposed network consequently enables more accurate predictions—in the presence of observational noise—in sequence modelling tasks such as timeseries forecasting that leverage the model interpretability, fast training, and function approximation properties of the RBF network.