Reviewerly/exHarmony huggingface dataset:
RIGA: Covert and Robust White-Box Watermarking of Deep Neural Networks

Watermarking of deep neural networks (DNN) can enable their tracing once released by a data owner to an online platform. In this paper, we generalize white-box watermarking algorithms for DNNs, where the data owner needs white-box access to the model to extract the watermark. White-box watermarking algorithms have the advantage that they do not impact the accuracy of the watermarked model. We propose Robust whIte-box GAn watermarking (RIGA), a novel white-box watermarking algorithm that uses adversarial training. Our extensive experiments demonstrate that the proposed watermarking algorithm not only does not impact accuracy, but also significantly improves the covertness and robustness over the current state-of-art.
SoK: All You Ever Wanted to Know About x86/x64 Binary Disassembly But Were Afraid to Ask

Disassembly of binary code is hard, but necessary for improving the security of binary software. Over the past few decades, research in binary disassembly has produced many tools and frameworks, which have been made available to researchers and security professionals. These tools employ a variety of strategies that grant them different characteristics. The lack of systematization, however, impedes new research in the area and makes selecting the right tool hard, as we do not understand the strengths and weaknesses of existing tools. In this paper, we systematize binary disassembly through the study of nine popular, open-source tools. We couple the manual examination of their code bases with the most comprehensive experimental evaluation (thus far) using 3,788 binaries. Our study yields a comprehensive description and organization of strategies for disassembly, classifying them as either algorithm or else heuristic. Meanwhile, we measure and report the impact of individual algorithms on the results of each tool. We find that while principled algorithms are used by all tools, they still heavily rely on heuristics to increase code coverage. Depending on the heuristics used, different coverage-vs-correctness trade-offs come in play, leading to tools with different strengths and weaknesses. We envision that these findings will help users pick the right tool and assist researchers in improving binary disassembly.
Twitter Data Sentiment Analysis Using Naive Bayes Classifier and Generation of Heat Map for Analyzing Intensity Geographically

Analysis of tweets or microblogs posted by users of a popular microblogging service, Twitter, using natural language processing is a very powerful tool to understand public sentiment on any worldly issue. This work expounds a hybrid approach using both corpus-based and dictionary-based methods to determine the semantic orientation of the opinion words in tweets using Naïve Bayes classifier. The accuracy of the classification has been verified with the use of Weka Tool. The valuable knowledge gained should be very informative and easy to understand. The proposed system intends to do that and also formulate a way to plot this Twitter data on Google Maps. A heat map showcases the intensity of a particular sentiment from different locations in the world. The actual implementation of this project has been done on Python language using various libraries for specific tasks. We have utilized ‘TextBlob’ library for implementing Naïve Bayes classifier.
Boosting Inertial-Based Human Activity Recognition With Transformers

Activity recognition problems such as human activity recognition and smartphone location recognition can improve the accuracy of different navigation or healthcare tasks, which rely solely on inertial sensors. Current learning-based approaches for activity recognition from inertial data employ convolutional neural networks or long short term memory architectures. Recently, Transformers were shown to outperform these architectures for sequence analysis tasks. This work presents an activity recognition model based on Transformers which offers an improved and general framework for learning activity recognition tasks. For evaluation purposes, several datasets, with more than 27 hours of inertial data recordings collected by 91 users, are employed. Those datasets represent different user activity scenarios with varying difficulty. The proposed approach consistently achieves better accuracy and generalizes better across all examined datasets and scenarios. A codebase implementing the described framework is available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/yolish/har-with-imu-transformer</uri> .
Survey on reinforcement learning for language processing

In recent years some researchers have explored the use of reinforcement learning (RL) algorithms as key components in the solution of various natural language processing tasks. For instance, some of these algorithms leveraging deep neural learning have found their way into conversational systems. This paper reviews the state of the art of RL methods for their possible use for different problems of natural language processing, focusing primarily on conversational systems, mainly due to their growing relevance. We provide detailed descriptions of the problems as well as discussions of why RL is well-suited to solve them. Also, we analyze the advantages and limitations of these methods. Finally, we elaborate on promising research directions in natural language processing that might benefit from reinforcement learning.
AnimePose: Multi-person 3D pose estimation and animation

3D animation of human body movement is quite challenging as it involves using a huge setup with several motion trackers all over the persons body to track the movements of every limb. This is time- consuming and may cause the person discomfort in wearing exoskeleton body suits with motion sensors. In this work, we present a trivial yet effective solution to generate simple 3D animation of human movement of multiple persons from a 2D video using deep learning. Although significant improvement has been achieved recently in 3D human pose estimation, most of the prior works work well in case of single person pose estimation and multi-person pose estimation is still a challenging problem. In this work, we firstly propose a supervised multi-person 3D pose estimation and animation framework namely AnimePose for a given input RGB video sequence. The pipeline of the proposed system consists of various modules: i) Multi-Person 2D pose estimation, ii) Depth Map estimation, iii) Lifting 2D poses to 3D poses, iv) Person trajectory prediction and human pose tracking. Our proposed system produces comparable results on previous state-of-the-art 3D multi-person pose estimation methods on publicly available dataset MuPoTS-3D dataset and it also outperforms previous competing human pose tracking methods by a significant margin of 11.7% performance gain on MOTA score on Posetrack 2018 dataset.
Class-Imbalanced Deep Learning via a Class-Balanced Ensemble

Class imbalance is a prevalent phenomenon in various real-world applications and it presents significant challenges to model learning, including deep learning. In this work, we embed ensemble learning into the deep convolutional neural networks (CNNs) to tackle the class-imbalanced learning problem. An ensemble of auxiliary classifiers branching out from various hidden layers of a CNN is trained together with the CNN in an end-to-end manner. To that end, we designed a new loss function that can rectify the bias toward the majority classes by forcing the CNN's hidden layers and its associated auxiliary classifiers to focus on the samples that have been misclassified by previous layers, thus enabling subsequent layers to develop diverse behavior and fix the errors of previous layers in a batch-wise manner. A unique feature of the new method is that the ensemble of auxiliary classifiers can work together with the main CNN to form a more powerful combined classifier, or can be removed after finished training the CNN and thus only acting the role of assisting class imbalance learning of the CNN to enhance the neural network's capability in dealing with class-imbalanced data. Comprehensive experiments are conducted on four benchmark data sets of increasing complexity (CIFAR-10, CIFAR-100, iNaturalist, and CelebA) and the results demonstrate significant performance improvements over the state-of-the-art deep imbalance learning methods.
Model-Free Quantum Control with Reinforcement Learning

A machine-learning agent learns how to control a quantum system---a key element of any quantum technology without any prior knowledge---by studying how a real quantum system responds to various actions.
Oriented Bounding Boxes for Small and Freely Rotated Objects

A novel object detection method is presented that handles freely rotated objects of arbitrary sizes, including tiny objects as small as $2\times 2$ pixels. Such tiny objects appear frequently in remotely sensed images, and present a challenge to recent object detection algorithms. More importantly, current object detection methods have been designed originally to accommodate axis-aligned bounding box detection, and therefore fail to accurately localize oriented boxes that best describe freely rotated objects. In contrast, the proposed CNN-based approach uses potential pixel information at multiple scale levels without the need for any external resources, such as anchor boxes.The method encodes the precise location and orientation of features of the target objects at grid cell locations. Unlike existing methods which regress the bounding box location and dimension,the proposed method learns all the required information by classification, which has the added benefit of enabling oriented bounding box detection without any extra computation. It thus infers the bounding boxes only at inference time by finding the minimum surrounding box for every set of the same predicted class labels. Moreover, a rotation-invariant feature representation is applied to each scale, which imposes a regularization constraint to enforce covering the 360 degree range of in-plane rotation of the training samples to share similar features. Evaluations on the xView and DOTA datasets show that the proposed method uniformly improves performance over existing state-of-the-art methods.
Definition of the Future Skills Needs of Job Profiles in the Renewable Energy Sector

The growth of the renewable energy industry is happening at a swift pace pushed, by the emergence of Industry 4.0. Smart technologies like artificial intelligence (AI), Big Data, the Internet of Things (IoT), Digital Twin (DT), etc. enable companies within the sector of renewable energies to drastically improve their operations. In this sectoral context, where upgraded sustainability standards also play a vital role, it is necessary to fulfil the human capital requirements of the imminent technological advances. This article aims to determine the current skills of the renewable energy industry workforce and to predict the upcoming skill requirements linked to a digital transition by creating a unified database that contains both types of skills. This will serve as a tool for renewable energy businesses, education centers, and policymakers to plan the training itinerary necessary to close the skills gap, as part of the sectoral strategy to achieve a competent future workforce.
A Hierarchical Transformation-Discriminating Generative Model for Few Shot Anomaly Detection

Anomaly detection, the task of identifying unusual samples in data, often relies on a large set of training samples. In this work, we consider the setting of few-shot anomaly detection in images, where only a few images are given at training. We devise a hierarchical generative model that captures the multi-scale patch distribution of each training image. We further enhance the representation of our model by using image transformations and optimize scale-specific patch-discriminators to distinguish between real and fake patches of the image, as well as between different transformations applied to those patches. The anomaly score is obtained by aggregating the patch-based votes of the correct transformation across scales and image regions. We demonstrate the superiority of our method on both the one-shot and few-shot settings, on the datasets of Paris, CIFAR10, MNIST and FashionMNIST as well as in the setting of defect detection on MVTec. In all cases, our method outperforms the recent baseline methods.
Towards achieving the best solution to utilize photovoltaic solar panels for residential buildings in urban areas

A way to find the best solution to utilize photovoltaic solar panels for residential buildings in urban areas is presented here. Three scenarios, namely, connecting to the grid with and without batteries, and full feed-in, are considered. After obtaining the optimal solution for each item, the best one is selected through a comprehensive comparison by considering energy, economic, and environmental criteria which are taken into account as the objective of the conducted multi-objective optimization. The combination of non-dominated sorting genetic algorithm II (NSGA-II) and technique for order preference by similarity to ideal solution (TOPSIS) is utilized for determination of the optimal answer for each scenario. A residential building complex in Tehran, Iran, as one of the biggest cities in the world is chosen as the case study. The results show that the full feed-in strategy offers the best condition among the alternatives. The optimal answer of this scenario enjoys the high performance ratio of 78.6 % (Performance ratio indicates the ratio of the actual to the theoretical energy output). Moreover, it provides levelized cost of electricity, payback period, and the internal rate of return of 0.15 $.(kWh)−1, 7.3 years, and 21.8 %, while it has the values of 487 $, 0.101 species.year, and 0.023 DALY for damage to resources, ecosystem, and human health, respectively.
Deep self-representative subspace clustering network

Deep learning based subspace clustering networks have been a significant technique for motion segmentation, unsupervised image segmentation, image representation and compression, and face clustering by separating the high-dimensional data points into their representative low-dimensional linear subspaces. Effective feature selection is critical to remove redundant samples and select the representative feature subset from high-dimensional data space; hence deriving the number of subspaces, their dimensions, data segmentation, and a basis for each subspace. The effective self-representative feature selection and emphasis by scaling the feature map in the learned embedded space is required for deep learning based subspace clustering to reduce the number of parameters and dimension of the self-representative layer. In this paper, we propose a self-representative feature extraction deep neural network for unsupervised subspace clustering to improve representativeness and clustering ability. The extensive relevant results on various data demonstrate that deep subspace clustering employing self-representative features from high-dimensional data can effectively reduce the dimension of the self-representative layer while improving performance.
Don’t Do That! Hunting Down Visual Design Smells in Complex UIs Against Design Guidelines

Just like code smells in source code, UI design has visual design smells. We study 93 don't-do-that guidelines in the Material Design, a complex design system created by Google. We find that these don't-guidelines go far beyond UI aesthetics, and involve seven general design dimensions (layout, typography, iconography, navigation, communication, color, and shape) and four component design aspects (anatomy, placement, behavior, and usage). Violating these guidelines results in visual design smells in UIs (or UI design smells). In a study of 60,756 UIs of 9,286 Android apps, we find that 7,497 UIs of 2,587 apps have at least one violation of some Material Design guidelines. This reveals the lack of developer training and tool support to avoid UI design smells. To fill this gap, we design an automated UI design smell detector (UIS-Hunter) that extracts and validates multi-modal UI information (component metadata, typography, iconography, color, and edge) for detecting the violation of diverse don't-guidelines in Material Design. The detection accuracy of UIS-Hunter is high (precision=0.81, recall=0.90) on the 60,756 UIs of 9,286 apps. We build a guideline gallery with real-world UI design smells that UIS-Hunter detects for developers to learn the best Material Design practices. Our user studies show that UIS-Hunter is more effective than manual detection of UI design smells, and the UI design smells that are detected by UIS-Hunter have severely negative impacts on app users.
Emotion Controllable Speech Synthesis Using Emotion-Unlabeled Dataset with the Assistance of Cross-Domain Speech Emotion Recognition

Neural text-to-speech (TTS) approaches generally require a huge number of high quality speech data, which makes it difficult to obtain such a dataset with extra emotion labels. In this paper, we propose a novel approach for emotional TTS synthesis on a TTS dataset without emotion labels. Specifically, our proposed method consists of a cross-domain speech emotion recognition (SER) model and an emotional TTS model. Firstly, we train the cross-domain SER model on both SER and TTS datasets. Then, we use emotion labels on the TTS dataset predicted by the trained SER model to build an auxiliary SER task and jointly train it with the TTS model. Experimental results show that our proposed method can generate speech with the specified emotional expressiveness and nearly no hindering on the speech quality.
“It cannot do all of my work”: Community Health Worker Perceptions of AI-Enabled Mobile Health Applications in Rural India

Recent advances in Artificial Intelligence (AI) suggest that AI applications could transform healthcare delivery in the Global South. However, as researchers and technology companies rush to develop AI applications that aid the health of marginalized communities, it is critical to consider the needs and perceptions of the community health workers (CHWs) who will have to integrate these AI applications into the essential healthcare services they provide to rural communities. We describe a qualitative study examining CHWs' perceptions of an AI application for automated disease diagnosis. Drawing on data from 21 interviews with CHWs in rural India, we characterize (1) CHWs' knowledge, perceptions, and understandings of AI; and (2) the benefits and challenges that CHWs anticipate as AI applications are integrated into their workflows, including their opinions on automation of their work, possible misdiagnosis and errors, data access and surveillance issues, security and privacy challenges, and questions concerning trust. We conclude by discussing the implications of our work for HCI and AI research in low-resource environments.
Adaptive memetic differential evolution with niching competition and supporting archive strategies for multimodal optimization

Multimodal optimization, which aims at locating multiple optimal solutions within the search space, is inherently a difficult problem.This work proposes an adaptive memetic differential evolution algorithm with niching competition and supporting archive strategies to tackle the problem.In the proposed algorithm, a niching competition strategy is designed to competitively employ niches according to their potentials by encouraging high potential niches for exploitation while low potential niches for exploration, thus appropriately searching the space to identify multiple optima.Further, a supporting archive strategy is devised and implemented at the niche level with a dual purpose of helping maintain potential optima as well as facilitate the evolution of population.In this strategy, the writing and reading of archive is implicitly implemented during evolution rather than requiring external rules.Additionally, an adaptive Cauchy-based local search scheme, which considers the possible locations of optima to implement the local search, is developed and incorporated into the proposed method to efficiently and properly improve niching seeds.The resulting algorithm has
Meaning and expected surfaces combine to guide attention during visual search in scenes

How do spatial constraints and meaningful scene regions interact to control overt attention during visual search for objects in real-world scenes? To answer this question, we combined novel surface maps of the likely locations of target objects with maps of the spatial distribution of scene semantic content. The surface maps captured likely target surfaces as continuous probabilities. Meaning was represented by meaning maps highlighting the distribution of semantic content in local scene regions. Attention was indexed by eye movements during the search for target objects that varied in the likelihood they would appear on specific surfaces. The interaction between surface maps and meaning maps was analyzed to test whether fixations were directed to meaningful scene regions on target-related surfaces. Overall, meaningful scene regions were more likely to be fixated if they appeared on target-related surfaces than if they appeared on target-unrelated surfaces. These findings suggest that the visual system prioritizes meaningful scene regions on target-related surfaces during visual search in scenes.
You’re Making Me Sick

While multiple studies suggest that female-identified participants are more likely to experience cybersickness in virtual reality (VR), our systematic review of 71 eligible VR publications (59 studies and 12 surveys) pertaining to gender and cybersickness reveals a number of confounding factors in study design (e.g., a variety of technical specifications, tasks, content), a lack of demographic data, and a bias in participant recruitment. Our review shows an ongoing need within VR research to more consistently include and report on women’s experiences in VR to better understand the gendered possibility of cybersickness. Based on the gaps identified in our systematic review, we contribute study design recommendations for future work, arguing that gender considerations are necessary at every stage of VR study design, even when the study is not ‘about’ gender.
BLOCKEYE: Hunting for DeFi Attacks on Blockchain

Decentralized finance, i.e., DeFi, has become the most popular type of application on many public blockchains (e.g., Ethereum) in recent years. Compared to the traditional finance, DeFi allows customers to flexibly participate in diverse blockchain financial services (e.g., lending, borrowing, collateralizing, exchanging etc.) via smart contracts at a relatively low cost of trust. However, the open nature of DeFi inevitably introduces a large attack surface, which is a severe threat to the security of participantsfunds. In this paper, we proposed BLOCKEYE, a real-time attack detection system for DeFi projects on the Ethereum blockchain. Key capabilities provided by BLOCKEYE are twofold: (1) Potentially vulnerable DeFi projects are identified based on an automatic security analysis process, which performs symbolic reasoning on the data flow of important service states, e.g., asset price, and checks whether they can be externally manipulated. (2) Then, a transaction monitor is installed offchain for a vulnerable DeFi project. Transactions sent not only to that project but other associated projects as well are collected for further security analysis. A potential attack is flagged if a violation is detected on a critical invariant configured in BLOCKEYE, e.g., Benefit is achieved within a very short time and way much bigger than the cost. We applied BLOCKEYE in several popular DeFi projects and managed to discover potential security attacks that are unreported before. A video of BLOCKEYE is available at https://youtu.be/7DjsWBLdlQU.