OpenRaiser/Intern-Atlas huggingface dataset:
Human-centered Explainable AI: Towards a Reflective Sociotechnical Approach
Venue: arXiv cs.HC 2020
Year: 2020

Explanations--a form of post-hoc interpretability--play an instrumental role in making systems accessible as AI continues to proliferate complex and sensitive sociotechnical systems. In this paper, we introduce Human-centered Explainable AI (HCXAI) as an approach that puts the human at the center of technology design. It develops a holistic understanding of "who" the human is by considering the interplay of values, interpersonal dynamics, and the socially situated nature of AI systems. In particular, we advocate for a reflective sociotechnical approach. We illustrate HCXAI through a case study of an explanation system for non-technical end-users that shows how technical advancements and the understanding of human factors co-evolve. Building on the case study, we lay out open research questions pertaining to further refining our understanding of "who" the human is and extending beyond 1-to-1 human-computer interactions. Finally, we propose that a reflective HCXAI paradigm-mediated through the perspective of Critical Technical Practice and supplemented with strategies from HCI, such as value-sensitive design and participatory design--not only helps us understand our intellectual blind spots, but it can also open up new design and research spaces.
Selective Segmentation Networks Using Top-Down Attention
Venue: arXiv cs.CV 2020
Year: 2020

Convolutional neural networks model the transformation of the input sensory data at the bottom of a network hierarchy to the semantic information at the top of the visual hierarchy. Feedforward processing is sufficient for some object recognition tasks. Top-Down selection is potentially required in addition to the Bottom-Up feedforward pass. It can, in part, address the shortcoming of the loss of location information imposed by the hierarchical feature pyramids. We propose a unified 2-pass framework for object segmentation that augments Bottom-Up \convnets with a Top-Down selection network. We utilize the top-down selection gating activities to modulate the bottom-up hidden activities for segmentation predictions. We develop an end-to-end multi-task framework with loss terms satisfying task requirements at the two ends of the network. We evaluate the proposed network on benchmark datasets for semantic segmentation, and show that networks with the Top-Down selection capability outperform the baseline model. Additionally, we shed light on the superior aspects of the new segmentation paradigm and qualitatively and quantitatively support the efficiency of the novel framework over the baseline model that relies purely on parametric skip connections.
Neural Oscillations for Encoding and Decoding Declarative Memory using EEG Signals
Venue: arXiv cs.NE 2020
Year: 2020

Declarative memory has been studied for its relationship with remembering daily life experiences. Previous studies reported changes in power spectra during encoding phase related to behavioral performance, however decoding phase still needs to be explored. This study investigates neural oscillations changes related to memory process. Participants were asked to perform a memory task for encoding and decoding phase while EEG signals were recorded. Results showed that for encoding phase, there was a significant decrease of power in low beta, high beta bands over fronto-central area and a decrease in low beta, high beta and gamma bands over left temporal area related to successful subsequent memory effects. For decoding phase, only significant decreases of alpha power were observed over fronto-central area. This finding showed relevance of beta and alpha band for encoding and decoding phase of a memory task respectively.
Adversarially Robust Frame Sampling with Bounded Irregularities
Venue: arXiv cs.CV 2020
Year: 2020

In recent years, video analysis tools for automatically extracting meaningful information from videos are widely studied and deployed. Because most of them use deep neural networks which are computationally expensive, feeding only a subset of video frames into such algorithms is desired. Sampling the frames with fixed rate is always attractive for its simplicity, representativeness, and interpretability. For example, a popular cloud video API generated video and shot labels by processing only the first frame of every second in a video. However, one can easily attack such strategies by placing chosen frames at the sampled locations. In this paper, we present an elegant solution to this sampling problem that is provably robust against adversarial attacks and introduces bounded irregularities as well.
Removing Connected Obstacles in the Plane is FPT
Venue: arXiv cs.DS 2020
Year: 2020

Given two points in the plane, a set of obstacles defined by closed curves, and an integer $k$, does there exist a path between the two designated points intersecting at most $k$ of the obstacles? This is a fundamental and well-studied problem arising naturally in computational geometry, graph theory, wireless computing, and motion planning. It remains $\textsf{NP}$-hard even when the obstacles are very simple geometric shapes (e.g., unit-length line segments). In this paper, we show that the problem is fixed-parameter tractable ($\textsf{FPT}$) parameterized by $k$, by giving an algorithm with running time $k^{O(k^3)}n^{O(1)}$. Here $n$ is the number connected areas in the plane drawing of all the obstacles.
CoVoST: A Diverse Multilingual Speech-To-Text Translation Corpus
Venue: arXiv cs.CL 2020
Year: 2020

Spoken language translation has recently witnessed a resurgence in popularity, thanks to the development of end-to-end models and the creation of new corpora, such as Augmented LibriSpeech and MuST-C. Existing datasets involve language pairs with English as a source language, involve very specific domains or are low resource. We introduce CoVoST, a multilingual speech-to-text translation corpus from 11 languages into English, diversified with over 11,000 speakers and over 60 accents. We describe the dataset creation methodology and provide empirical evidence of the quality of the data. We also provide initial benchmarks, including, to our knowledge, the first end-to-end many-to-one multilingual models for spoken language translation. CoVoST is released under CC0 license and free to use. We also provide additional evaluation data derived from Tatoeba under CC licenses.
The Gaussian lossy Gray-Wyner network
Venue: arXiv cs.IT 2020
Year: 2020

We consider the problem of source coding subject to a fidelity criterion for the Gray-Wyner network that connects a single source with two receivers via a common channel and two private channels. The pareto-optimal trade-offs between the sum-rate of the private channels and the rate of the common channel is completely characterized for jointly Gaussian sources subject to the mean-squared error criterion, leveraging convex duality and an argument involving the factorization of convex envelopes. Specifically, it is attained by selecting the auxiliary random variable to be jointly Gaussian with the sources.
A Scalable IoT-Fog Framework for Urban Sound Sensing
Venue: arXiv cs.NI 2020
Year: 2020

Internet of Things (IoT) is a system of interrelated devices that can be used to allow large-scale collection and analysis of data. However, as it grew, IoT networks were not capable of managing the data from these services. As a result, cloud computing was introduced to address the need for datacentres for IoT networks. As the technology evolved, the demand for a proper means of supporting and managing crowdsensing and real-time data increased, and cloud servers could no longer keep up with the large volumes of incoming data. This demand brought rise to fog computing. It became an extension to the cloud and allowed resources to be allocated around the network effectively. Its integration to IoT reduced the strain towards the cloud servers. However, issues in high power consumption at the end device and data management constraints surfaced. This paper proposes two approaches to alleviate these issues to keep fog computing remain as a reliable option for IoT-related applications. We created an IoT-based sensing framework that used an urban sound classification model. Through active low and high power states and resource reallocation, we created a network configuration. We tested this configuration against IoT frameworks that use the default fog and cloud setups. The results improved the framework's end device power consumption and server latency. Overall, with the proposed framework, fog computing was proven to be capable of supporting a scalable IoT framework for urban sound sensing.
Encryption Algorithm for TCP Session Hijacking
Venue: arXiv cs.CR 2020
Year: 2020

Distributed network of the computer and the design defects of the TCP protocol are given to the network attack to be multiplicative. Based on the simple and open assumptions of the TCP protocol in academic and collaborative communication environments, the protocol lacks secure authentication. In this paper, by adding RSA-based cryptography technology, RSA-based signature technology, DH key exchange algorithm, and HAMC-SHA1 integrity verification technology to the TCP protocol, and propose a security strategy which can effectively defend against TCP session hijacking.
A Prototype of Serverless Lucene
Venue: arXiv cs.IR 2020
Year: 2020

This paper describes a working prototype that adapts Lucene, the world's most popular and most widely deployed open-source search library, to operate within a serverless environment in the cloud. Although the serverless search concept is not new, this work represents a substantial improvement over a previous implementation in eliminating most custom code and in enabling interactive search. While there remain limitations to the design, it nevertheless challenges conventional thinking about search architectures for particular operating points.
Action Graphs: Weakly-supervised Action Localization with Graph Convolution Networks
Venue: arXiv cs.CV 2020
Year: 2020

We present a method for weakly-supervised action localization based on graph convolutions. In order to find and classify video time segments that correspond to relevant action classes, a system must be able to both identify discriminative time segments in each video, and identify the full extent of each action. Achieving this with weak video level labels requires the system to use similarity and dissimilarity between moments across videos in the training data to understand both how an action appears, as well as the sub-actions that comprise the action's full extent. However, current methods do not make explicit use of similarity between video moments to inform the localization and classification predictions. We present a novel method that uses graph convolutions to explicitly model similarity between video moments. Our method utilizes similarity graphs that encode appearance and motion, and pushes the state of the art on THUMOS '14, ActivityNet 1.2, and Charades for weakly supervised action localization.
Measuring the Utilization of Public Open Spaces by Deep Learning: a Benchmark Study at the Detroit Riverfront
Venue: arXiv cs.CV 2020
Year: 2020

Physical activities and social interactions are essential activities that ensure a healthy lifestyle. Public open spaces (POS), such as parks, plazas and greenways, are key environments that encourage those activities. To evaluate a POS, there is a need to study how humans use the facilities within it. However, traditional approaches to studying use of POS are manual and therefore time and labor intensive. They also may only provide qualitative insights. It is appealing to make use of surveillance cameras and to extract user-related information through computer vision. This paper proposes a proof-of-concept deep learning computer vision framework for measuring human activities quantitatively in POS and demonstrates a case study of the proposed framework using the Detroit Riverfront Conservancy (DRFC) surveillance camera network. A custom image dataset is presented to train the framework; the dataset includes 7826 fully annotated images collected from 18 cameras across the DRFC park space under various illumination conditions. Dataset analysis is also provided as well as a baseline model for one-step user localization and activity recognition. The mAP results are 77.5\% for {\it pedestrian} detection and 81.6\% for {\it cyclist} detection. Behavioral maps are autonomously generated by the framework to locate different POS users and the average error for behavioral localization is within 10 cm.
Seeing through the smoke : a world-wide comparative study of e-cigarette flavors, brands and markets using data from Reddit and Twitter
Venue: arXiv cs.SI 2020
Year: 2020

The growing popularity of E-cigarettes, an alternative to cigarettes, has motivated us to study trends of the brands, flavors and online market activity using posts from Reddit and Twitter. The main motivation for this world-wide study is to emphasize the difference that laws and regulations have on the usage and availability of different flavors and brands of vapes in different countries. Data has been obtained from subreddits belonging to e-cigarette communities from Australia, Canada, Europe, and the UK. Extensive cleaning of data, and rigorous text mining operations provide varying results for different countries. Varying results have been obtained from Reddit and Twitter since they provide different atmospheres to the users.
Joint Optimization of AI Fairness and Utility: A Human-Centered Approach
Venue: arXiv cs.AI 2020
Year: 2020

Today, AI is increasingly being used in many high-stakes decision-making applications in which fairness is an important concern. Already, there are many examples of AI being biased and making questionable and unfair decisions. The AI research community has proposed many methods to measure and mitigate unwanted biases, but few of them involve inputs from human policy makers. We argue that because different fairness criteria sometimes cannot be simultaneously satisfied, and because achieving fairness often requires sacrificing other objectives such as model accuracy, it is key to acquire and adhere to human policy makers' preferences on how to make the tradeoff among these objectives. In this paper, we propose a framework and some exemplar methods for eliciting such preferences and for optimizing an AI model according to these preferences.
Investigation of the Sense of Agency in Social Cognition, based on frameworks of Predictive Coding and Active Inference: A simulation study on multimodal imitative interaction
Venue: arXiv cs.RO 2020
Year: 2020

When agents interact socially with different intentions, conflicts are difficult to avoid. Although how agents can resolve such problems autonomously has not been determined, dynamic characteristics of agency may shed light on underlying mechanisms. The current study focused on the sense of agency (SoA), a specific aspect of agency referring to congruence between the agent's intention in acting and the outcome. Employing predictive coding and active inference as theoretical frameworks of perception and action generation, we hypothesize that regulation of complexity in the evidence lower bound of an agent's model should affect the strength of the agent's SoA and should have a critical impact on social interactions. We built a computational model of imitative interaction between a robot and a human via visuo-proprioceptive sensation with a variational Bayes recurrent neural network, and simulated the model in the form of pseudo-imitative interaction using recorded human body movement data. A key feature of the model is that each modality's complexity can be regulated differently with a hyperparameter assigned to each module. We first searched for an optimal setting that endows the model with appropriate coordination of multimodal sensation. This revealed that the vision module's complexity should be more tightly regulated than that of the proprioception module. Using the optimally trained model, we examined how changing the tightness of complexity regulation after training affects the strength of the SoA during interactions. The results showed that with looser regulation, an agent tends to act more egocentrically, without adapting to the other. In contrast, with tighter regulation, the agent tends to follow the other by adjusting its intention. We conclude that the tightness of complexity regulation crucially affects the strength of the SoA and the dynamics of interactions between agents.
Identification of Indian Languages using Ghost-VLAD pooling
Venue: arXiv cs.CL 2020
Year: 2020

In this work, we propose a new pooling strategy for language identification by considering Indian languages. The idea is to obtain utterance level features for any variable length audio for robust language recognition. We use the GhostVLAD approach to generate an utterance level feature vector for any variable length input audio by aggregating the local frame level features across time. The generated feature vector is shown to have very good language discriminative features and helps in getting state of the art results for language identification task. We conduct our experiments on 635Hrs of audio data for 7 Indian languages. Our method outperforms the previous state of the art x-vector [11] method by an absolute improvement of 1.88% in F1-score and achieves 98.43% F1-score on the held-out test data. We compare our system with various pooling approaches and show that GhostVLAD is the best pooling approach for this task. We also provide visualization of the utterance level embeddings generated using Ghost-VLAD pooling and show that this method creates embeddings which has very good language discriminative features.
Age of Information in a Decentralized Network of Parallel Queues with Routing and Packets Losses
Venue: arXiv cs.PF 2020
Year: 2020

The paper deals with Age of Information (AoI) in a network of multiple sources and parallel queues with buffering capabilities, preemption in service and losses in served packets. The queues do not communicate between each other and the packets are dispatched through the queues according to a predefined probabilistic routing. By making use of the Stochastic Hybrid System (SHS) method, we provide a derivation of the average AoI of a system of two parallel queues (with and without buffer capabilities) and compare the results with those of a single queue. We show that known results of packets delay in Queuing Theory do not hold for the AoI. Unfortunately, the complexity of computing the average AoI using the SHS method increases highly with the number of queues. We therefore provide an upper bound of the average AoI in a system of an arbitrary number of M/M/1/(N+1) queues and show its tightness in various regimes. This upper bound allows providing a tight approximation of the average AoI with a very low complexity. We then provide a game framework that allows each source to determine its best probabilistic routing decision. By using Mean Field Games, we provide an analysis of the routing game framework, propose an efficient iterative method to find the routing decision of each source and prove its convergence to the desired equilibrium.
Geocoding of trees from street addresses and street-level images
Venue: arXiv cs.CV 2020
Year: 2020

We introduce an approach for updating older tree inventories with geographic coordinates using street-level panorama images and a global optimization framework for tree instance matching. Geolocations of trees in inventories until the early 2000s where recorded using street addresses whereas newer inventories use GPS. Our method retrofits older inventories with geographic coordinates to allow connecting them with newer inventories to facilitate long-term studies on tree mortality etc. What makes this problem challenging is the different number of trees per street address, the heterogeneous appearance of different tree instances in the images, ambiguous tree positions if viewed from multiple images and occlusions. To solve this assignment problem, we (i) detect trees in Google street-view panoramas using deep learning, (ii) combine multi-view detections per tree into a single representation, (iii) and match detected trees with given trees per street address with a global optimization approach. Experiments for > 50000 trees in 5 cities in California, USA, show that we are able to assign geographic coordinates to 38 % of the street trees, which is a good starting point for long-term studies on the ecosystem services value of street trees at large scale.
I Know Where You Are Coming From: On the Impact of Social Media Sources on AI Model Performance
Venue: arXiv cs.IR 2020
Year: 2020

Nowadays, social networks play a crucial role in human everyday life and no longer purely associated with spare time spending. In fact, instant communication with friends and colleagues has become an essential component of our daily interaction giving a raise of multiple new social network types emergence. By participating in such networks, individuals generate a multitude of data points that describe their activities from different perspectives and, for example, can be further used for applications such as personalized recommendation or user profiling. However, the impact of the different social media networks on machine learning model performance has not been studied comprehensively yet. Particularly, the literature on modeling multi-modal data from multiple social networks is relatively sparse, which had inspired us to take a deeper dive into the topic in this preliminary study. Specifically, in this work, we will study the performance of different machine learning models when being learned on multi-modal data from different social networks. Our initial experimental results reveal that social network choice impacts the performance and the proper selection of data source is crucial.
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters
Venue: arXiv cs.CL 2020
Year: 2020

We study the problem of injecting knowledge into large pre-trained models like BERT and RoBERTa. Existing methods typically update the original parameters of pre-trained models when injecting knowledge. However, when multiple kinds of knowledge are injected, the historically injected knowledge would be flushed away. To address this, we propose K-Adapter, a framework that retains the original parameters of the pre-trained model fixed and supports the development of versatile knowledge-infused model. Taking RoBERTa as the backbone model, K-Adapter has a neural adapter for each kind of infused knowledge, like a plug-in connected to RoBERTa. There is no information flow between different adapters, thus multiple adapters can be efficiently trained in a distributed way. As a case study, we inject two kinds of knowledge in this work, including (1) factual knowledge obtained from automatically aligned text-triplets on Wikipedia and Wikidata and (2) linguistic knowledge obtained via dependency parsing. Results on three knowledge-driven tasks, including relation classification, entity typing, and question answering, demonstrate that each adapter improves the performance and the combination of both adapters brings further improvements. Further analysis indicates that K-Adapter captures versatile knowledge than RoBERTa.