Reviewerly/exHarmony huggingface dataset:
Multi-layer map: Augmenting semantic visual memory

The modern view of things in the science of robotics imposes that when working in a human environment, understanding of its equivalent semantics is required. In this paper, we present a graph-based unsupervised semantic clustering method and a novel cluster matching technique, with a view to create a multi-layer semantic memory map robust to illumination changes. Using indoor data collected by an unmanned aerial robot (UAR) and a publicly available dataset, we apply a community detection algorithm (CDA) to find efficiently coherent visual data throughout the trajectory creating a semantic base map. Then, we optimize the formed communities using metric information by implementing an hierarchical agglomerative clustering algorithm. The multilayer semantic map is created by constructing map instances for variant lighting conditions and matching the generated clusters to their base map correspondence. The proposed matching method relies on the graphs centrality indicators to identify central images of a region and utilize them to efficiently extract resemblances within the base map.
Conversion of <scp>Landsat</scp> 8 multispectral data through modified private content‐based image retrieval technique for secure transmission and privacy

Abstract In this research work, we have developed a new image encryption‐based algorithm for Landsat 8 satellite images. Image statistical parameters related to first‐order image statistics, that is, mean (μ), SD ( σ) , and variance ( σ 2 ) are used to obtain the feature values for both original images and their encrypted versions. Multispectral satellite images generally contain a wide range of bands, which is 9 to 11 specifically for the Landsat 8 satellite. Hence, the information content of these images is richer than typical images. Through this algorithm, we aim to reduce the image content by randomly selecting only 10% of the total image pixels. These pixels are used to create new images, the so‐called encrypted images. The visual presentation of the encrypted image resembles a noisy picture. First‐order image statistics are calculated for both original images and their encrypted counterparts. These parameters show a similarity in their numerical values for both original and encrypted images. Thus, noisy pictures can be used for transmission in place of the original image when an outcome is decided through statistical image features. The visual similarity in the original images and encrypted images is obtained from the histogram signature plot for both original and encrypted images. The proposed algorithm can hence be used to convert image data for secure transmission and privacy.
Lyra 2: Designing Interactive Visualizations by Demonstration

Recent graphical interfaces offer direct manipulation mechanisms for authoring visualizations, but are largely restricted to static output. To author interactive visualizations, users must instead turn to textual specification, but such approaches impose a higher technical burden. To bridge this gap, we introduce Lyra 2, a system that extends a prior visualization design environment with novel methods for authoring interaction techniques by demonstration. Users perform an interaction (e.g., button clicks, drags, or key presses) directly on the visualization they are editing. The system interprets this performance using a set of heuristics and enumerates suggestions of possible interaction designs. These heuristics account for the properties of the interaction (e.g., target and event type) as well as the visualization (e.g., mark and scale types, and multiple views). Interaction design suggestions are displayed as thumbnails; users can preview and test these suggestions, iteratively refine them through additional demonstrations, and finally apply and customize them via property inspectors. We evaluate our approach through a gallery of diverse examples, and evaluate its usability through a first-use study and via an analysis of its cognitive dimensions. We find that, in Lyra 2, interaction design by demonstration enables users to rapidly express a wide range of interactive visualizations.
Illicit drug prices and quantity discounts: A comparison between a cryptomarket, social media, and police data

Illicit drugs are increasingly sold on cryptomarkets and on social media. Buyers and sellers perceive these online transactions as less risky than conventional street-level exchanges. Following the Risks & Prices framework, law enforcement is the largest cost component of illicit drug distribution. We examine whether prices on cryptomarkets are lower than prices on social media and prices reported by law enforcement on primarily offline markets. Data consists of online advertisements for illicit drugs in Sweden in 2018, scraped from the cryptomarket Flugsvamp 2.0 (n = 826) and collected with digital ethnography on Facebook (n = 446). Observations are advertisements for herbal cannabis (n = 421), cannabis resin, hash (n = 594), and cocaine (n = 257) from 156 sellers. Prices are compared with estimates from Swedish police districts (n = 53). Three multilevel linear regression models are estimated, one for each drug type, comparing price levels and discount elasticities for each platform and between sellers on each platform. Price levels are similar on the two online platforms, but cocaine is slightly more expensive on social media. There are quantity discounts for all three drug types on both platforms with coefficients between -0.10 and -0.21. Despite the higher competition between sellers on cryptomarkets, prices are not lower compared to social media. Online price levels for hash and cocaine are similar to those reported by police at the 1 g level. Mean prices and quantity discounts are similar in the two online markets. This provides support for the notion that research on cryptomarkets can also inform drug market analysis in a broader sense. Online advertisements for drugs constitute a new detailed transaction-level data source for supply-side price information for research.
Visual Analysis of Discrimination in Machine Learning

The growing use of automated decision-making in critical applications, such as crime prediction and college admission, has raised questions about fairness in machine learning. How can we decide whether different treatments are reasonable or discriminatory? In this paper, we investigate discrimination in machine learning from a visual analytics perspective and propose an interactive visualization tool, DiscriLens, to support a more comprehensive analysis. To reveal detailed information on algorithmic discrimination, DiscriLens identifies a collection of potentially discriminatory itemsets based on causal modeling and classification rules mining. By combining an extended Euler diagram with a matrix-based visualization, we develop a novel set visualization to facilitate the exploration and interpretation of discriminatory itemsets. A user study shows that users can interpret the visually encoded information in DiscriLens quickly and accurately. Use cases demonstrate that DiscriLens provides informative guidance in understanding and reducing algorithmic discrimination.
Encoding Visual Attributes in Capsules for Explainable Medical Diagnoses

Convolutional neural network based systems have largely failed to be adopted in many high-risk application areas, including healthcare, military, security, transportation, finance, and legal, due to their highly uninterpretable "black-box" nature. Towards solving this deficiency, we teach a novel multi-task capsule network to improve the explainability of predictions by embodying the same high-level language used by human-experts. Our explainable capsule network, X-Caps, encodes high-level visual object attributes within the vectors of its capsules, then forms predictions based solely on these human-interpretable features. To encode attributes, X-Caps utilizes a new routing sigmoid function to independently route information from child capsules to parents. Further, to provide radiologists with an estimate of model confidence, we train our network on a distribution of expert labels, modeling inter-observer agreement and punishing over/under confidence during training, supervised by human-experts' agreement. X-Caps simultaneously learns attribute and malignancy scores from a multi-center dataset of over 1000 CT scans of lung cancer screening patients. We demonstrate a simple 2D capsule network can outperform a state-of-the-art deep dense dual-path 3D CNN at capturing visually-interpretable high-level attributes and malignancy prediction, while providing malignancy prediction scores approaching that of non-explainable 3D CNNs. To the best of our knowledge, this is the first study to investigate capsule networks for making predictions based on radiologist-level interpretable attributes and its applications to medical image diagnosis. Code is publicly available at https://github.com/lalonderodney/X-Caps.
UltraTrail: A Configurable Ultralow-Power TC-ResNet AI Accelerator for Efficient Keyword Spotting

Recent advances in machine learning show the superior behavior of temporal convolutional networks (TCNs) and especially their combination with residual networks (TC-ResNet) for intelligent sensor signal processing in comparison to classical CNNs and LSTMs. In this article, we propose UltraTrail, a configurable, ultralow-power TC-ResNet AI accelerator for sensor signal processing and its application to efficient keyword spotting (KWS). Following a strict hardware/model co-design approach, we have derived an optimized low-power hardware architecture for generalized TC-ResNet topologies consisting of a configurable array of processing elements and a distributed memory with dynamic content reallocation. We additionally extend the network with conditional computing to reduce the number of operations during inference and to provide the possibility for power-gating. The final accelerator implementation in Globalfoundries' 22FDX technology achieves a power consumption of 8.2 μW for the task of always-on KWS meeting the real-time requirement of 100 ms per inference with an accuracy of 93% on the Google Speech Command Dataset.
Automatic Operating Room Surgical Activity Recognition for Robot-Assisted Surgery

Automatic recognition of surgical activities in the operating room (OR) is a key technology for creating next generation intelligent surgical devices and workflow monitoring/support systems. Such systems can potentially enhance efficiency in the OR, resulting in lower costs and improved care delivery to the patients. In this paper, we investigate automatic surgical activity recognition in robot-assisted operations. We collect the first large-scale dataset including 400 full-length multi-perspective videos from a variety of robotic surgery cases captured using Time-of-Flight cameras. We densely annotate the videos with 10 most recognized and clinically relevant classes of activities. Furthermore, we investigate state-of-the-art computer vision action recognition techniques and adapt them for the OR environment and the dataset. First, we fine-tune the Inflated 3D ConvNet (I3D) for clip-level activity recognition on our dataset and use it to extract features from the videos. These features are then fed to a stack of 3 Temporal Gaussian Mixture layers which extracts context from neighboring clips, and eventually go through a Long Short Term Memory network to learn the order of activities in full-length videos. We extensively assess the model and reach a peak performance of \({\sim }88\%\) mean Average Precision.
Organizational Governance, Social Bonds and Information Security Policy Compliance: A Perspective towards Oil and Gas Employees

Information security attacks on oil and gas (O&amp;G) organizations have increased since the last decade. From 2015 to 2019, almost 70 percent of O&amp;G organizations faced at least one significant security breach worldwide. Research has shown that 43 percent of security attacks on O&amp;G organizations occur due to the non-compliant behavior of O&amp;G employees towards information security policy. The existing literature provides multiple solutions for technical security controls of O&amp;G organizations. However, there are very few studies available that address behavioral security controls, specifically for O&amp;G organizations of developing countries. The purpose of this study is to provide a comprehensive framework for information security policy compliance (ISPC) for the O&amp;G sector. A mixed-method approach is used to develop the research framework. Semi-structured interviews from O&amp;G specialists refined the developed framework. Based on qualitative study a survey questionnaire was developed. To evaluate the research framework, structural equation modeling was applied to a sample of 254 managers/executives from 150 Malaysian O&amp;G organizations. The obtained test results confirmed the proposed research model, according to which good social bonding among employees plays a critical role in improving ISPC. However, there was less support for the notion that all organizational governance factors significantly improve the social bonding of Malaysian O&amp;G organizations employees. This paper contributes to the current information system (IS) literature by exploring the interrelationships among organizational governance, social bonding, and information security policy compliance (ISPC) in Malaysian O&amp;G organizations.
Modeling both Intra- and Inter-modal Influence for Real-Time Emotion Detection in Conversations

Through much exploration in the past decade, emotion analysis in conversations was mainly conducted in textual scenario. Nowadays, with the popularization of speech and video communication, academia and industry have become gradually aware of the need in multimodal scenario. Therefore, emotion detection in conversations becomes increasingly hot not only in natural language processing (NLP) community but also in multimodal analysis community. Although previous studies normally argue that the emotion of current utterance in a conversation is much influenced by the content of historical utterances, their speakers and emotions, they model the influence derived from the history to the current utterance at the same granularity (Intra-modal influence). Intuitively, the clues of emotion detection may not exist in the history of the same modality as current utterance, but in the history of other modalities (Inter-modal influence). Besides, previous studies normally model the information propagation as the conversation flow. Intuitively, bidirectional modeling of information propagation in conversations provides rich clues for emotion detection. Therefore, this paper proposes a bidirectional dynamic dual influence network for real-time emotion detection in conversations, which can simultaneously model both intra- and inter-modal influence with bidirectional information propagation for current utterance and its historical utterances. Detailed experiments demonstrate that our approach much advances the state-of-the-art.
Elemental assessment of vegetation via portable X-ray fluorescence: Sample preparation and methodological considerations

This paper reviews the use of pXRF in elemental assessment of vegetation, with emphasis on the factors that affect pXRF readings, including water content, protective film effect, critical penetration depth of X-rays, sample thickness, measurement time, and particle size. An overview and guideline for pXRF analysis of vegetal matrixes were provided, aiming to minimize the errors and to improve the predictive accuracy of pXRF data.
Resource Efficient Domain Adaptation

Domain Adaptation (DA) aims at transferring knowledge from a labeled source domain to an unlabeled target domain. While re- markable advances have been witnessed recently, the power of DA methods still heavily depends on the network depth, especially when the domain discrepancy is large, posing an unprecedented challenge to DA in low-resource scenarios where fast and adaptive inference is required. How to bridge transferability and resource- efficient inference in DA becomes an important problem. In this paper, we propose Resource Efficient Domain Adaptation (REDA), a general framework that can adaptively adjust computation re- sources across 'easier' and 'harder' inputs. Based on existing multi- exit architectures, REDA has two novel designs: 1) Transferable distillation to distill the transferability of top classifier into the early exits; 2) Consistency weighting to control the distillation degree via prediction consistency. As a general method, REDA can be easily applied with a variety of DA methods. Empirical results and analy- ses justify that REDA can substantially improve the accuracy and accelerate the inference under domain shift and low resource.
I feel a moving crowd surrounds me: Exploring tactile feedback during immersive walking in a virtual crowd

Abstract The aim of our study was to investigate whether different tactile feedback conditions could affect the behavior of participants who were instructed to walk within a virtual reality environment surrounded by a virtual crowd of people. A road crossing scenario that takes place in a virtual metropolitan city was developed for this study. Participants were asked to walk toward the opposite sidewalk while wearing a tactile vest. At each road crossing, one of several tactile feedback conditions was generated, including No Tactile, Side Tactile, Back Tactile, Front Tactile, Accurate Tactile, and Random Tactile. During the virtual road crossing, the movement of the participants was captured, and movement‐related measurements were extracted and analyzed to evaluate the effects of tactile feedback on the participant's movement behavior. Additional data were collected through the distribution of a questionnaire to consider self‐reported ratings of the experimental conditions. The results revealed that tactile feedback conditions had significant effects on movement behavior, while the participants' ratings also indicated that they were affected by the tactile feedback conditions. We found that when the participants were immersed in a high‐density crowd simulation, they became sensitive to tactile feedback. However, they were not able to distinguish between the accurate feedback and the random feedback.
Trust in blockchains: Algorithmic and organizational

The recent emergence of blockchains has reconceptualized our understanding of crowdfunding, platforms, organization, and governance. The disruption, and disintermediation made possible by blockchains has rightfully received increasing attention from entrepreneurs, and entrepreneurship researchers alike. In this article, I broaden these perspectives by paying particular attention to trust in blockchains, and contrasting it with perspectives on trust from Transaction Cost Economics, Agency Theory, and the Resource Based View. In sum, I compare perspectives on trust in these traditional theories to the conception of trust in blockchains – a mix of algorithmic, and organizational – to offer new insights into the implications of the rise of blockchains.
Interpretable Embedding for Ad-Hoc Video Search

Answering query with semantic concepts has long been the mainstream approach for video search. Until recently, its performance is surpassed by concept-free approach, which embeds queries in a joint space as videos. Nevertheless, the embedded features as well as search results are not interpretable, hindering subsequent steps in video browsing and query reformulation. This paper integrates feature embedding and concept interpretation into a neural network for unified dual-task learning. In this way, an embedding is associated with a list of semantic concepts as an interpretation of video content. This paper empirically demonstrates that, by using either the embedding features or concepts, considerable search improvement is attainable on TRECVid benchmarked datasets. Concepts are not only effective in pruning false positive videos, but also highly complementary to concept-free search, leading to large margin of improvement compared to state-of-the-art approaches.
KM<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="d1e2545" altimg="si2.svg"><mml:msup><mml:mrow /><mml:mrow><mml:mn>4</mml:mn></mml:mrow></mml:msup></mml:math>: Visual reasoning via Knowledge Embedding Memory Model with Mutual Modulation

Visual reasoning is a special kind of visual question answering, which is essentially multi-step and compositional, and also requires intensive text-visual interaction. The most important and challenging problem of visual reasoning is to design an effective and robust visual reasoning model. To this end, there are two challenges to overcome. The first is that textual and visual information must be jointly considered to make accurate inferences about reasoning. The second is that existing deep learning-based works are often too specific to a particular task. To address these issues, we propose a knowledge memory embedding model with mutual modulation for visual reasoning. This approach learns not only knowledge-based embeddings derived from key–value memory network to make the full and joint of textual and visual information, but also exploits the prior knowledge to improve the performance with knowledge-based representation learning for applying other general reasoning tasks. Experimental results on four benchmarks show that the proposed approach significantly improves performance compared with other state-of-the-art methods, guarantees the robustness with our model. Most importantly, we apply our model to four reasoning tasks, and experimentally show that our model effectively supports relational reasoning and improves performance in several tasks and datasets.
Learning Tuple Compatibility for Conditional Outfit Recommendation

Outfit recommendation requires the answers of some challenging outfit compatibility questions such as 'Which pair of boots and school bag go well with my jeans and sweater?'. It is more complicated than conventional similarity search, and needs to consider not only visual aesthetics but also the intrinsic fine-grained and multi-category nature of fashion items. Some existing approaches solve the problem through sequential models or learning pair-wise distances between items. However, most of them only consider coarse category information in defining fashion compatibility while neglecting the fine-grained category information often desired in practical applications. To better define the fashion compatibility and more flexibly meet different needs, we propose a novel problem of learning compatibility among multiple tuples (each consisting of an item and category pair), and recommending fashion items following the category choices from customers. Our contributions include: 1) Designing a Mixed Category Attention Net (MCAN) which integrates both fine-grained and coarse category information into recommendation and learns the compatibility among fashion tuples. MCAN can explicitly and effectively generate diverse and controllable recommendations based on need. 2) Contributing a new dataset IQON, which follows eastern culture and can be used to test the generalization of recommendation systems. Our extensive experiments on a reference dataset Polyvore and our dataset IQON demonstrate that our method significantly outperforms state-of-the-art recommendation methods.