OpenRaiser/Intern-Atlas huggingface dataset:
A neural network walks into a lab: towards using deep nets as models for human behavior
Venue: arXiv cs.AI 2020
Year: 2020

What might sound like the beginning of a joke has become an attractive prospect for many cognitive scientists: the use of deep neural network models (DNNs) as models of human behavior in perceptual and cognitive tasks. Although DNNs have taken over machine learning, attempts to use them as models of human behavior are still in the early stages. Can they become a versatile model class in the cognitive scientist's toolbox? We first argue why DNNs have the potential to be interesting models of human behavior. We then discuss how that potential can be more fully realized. On the one hand, we argue that the cycle of training, testing, and revising DNNs needs to be revisited through the lens of the cognitive scientist's goals. Specifically, we argue that methods for assessing the goodness of fit between DNN models and human behavior have to date been impoverished. On the other hand, cognitive science might have to start using more complex tasks (including richer stimulus spaces), but doing so might be beneficial for DNN-independent reasons as well. Finally, we highlight avenues where traditional cognitive process models and DNNs may show productive synergy.
An Information-theoretic Visual Analysis Framework for Convolutional Neural Networks
Venue: arXiv cs.CV 2020
Year: 2020

Despite the great success of Convolutional Neural Networks (CNNs) in Computer Vision and Natural Language Processing, the working mechanism behind CNNs is still under extensive discussions and research. Driven by a strong demand for the theoretical explanation of neural networks, some researchers utilize information theory to provide insight into the black box model. However, to the best of our knowledge, employing information theory to quantitatively analyze and qualitatively visualize neural networks has not been extensively studied in the visualization community. In this paper, we combine information entropies and visualization techniques to shed light on how CNN works. Specifically, we first introduce a data model to organize the data that can be extracted from CNN models. Then we propose two ways to calculate entropy under different circumstances. To provide a fundamental understanding of the basic building blocks of CNNs (e.g., convolutional layers, pooling layers, normalization layers) from an information-theoretic perspective, we develop a visual analysis system, CNNSlicer. CNNSlicer allows users to interactively explore the amount of information changes inside the model. With case studies on the widely used benchmark datasets (MNIST and CIFAR-10), we demonstrate the effectiveness of our system in opening the blackbox of CNNs.
AR-Therapist: Design and Simulation of an AR-Game Environment as a CBT for Patients with ADHD
Venue: arXiv cs.HC 2020
Year: 2020

Attention Deficit Hyperactivity Disorder is one of the most common neurodevelopmental disorders in which patients have difficulties related to inattention, hyperactivity, and impulsivity. Those patients are in need of a psychological therapy use Cognitive Behavioral Therapy (CBT) to enhance the way they think and behave. This type of therapy is mostly common in treating patients with anxiety and depression but also is useful in treating autism, obsessive compulsive disorder and post-traumatic stress disorder. A major limitation of traditional CBT is that therapists may face difficulty in optimizing patients' neuropsychological stimulus following a specified treatment plan. Other limitations include availability, accessibility and level-of-experience of the therapists. Hence, this paper aims to design and simulate a generic cognitive model that can be used as an appropriate alternative treatment to traditional CBT, we term as "AR-Therapist." This model takes advantage of the current developments of augmented reality to engage patients in both real and virtual game-based environments.
Rolling-Unrolling LSTMs for Action Anticipation from First-Person Video
Venue: arXiv cs.CV 2020
Year: 2020

In this paper, we tackle the problem of egocentric action anticipation, i.e., predicting what actions the camera wearer will perform in the near future and which objects they will interact with. Specifically, we contribute Rolling-Unrolling LSTM, a learning architecture to anticipate actions from egocentric videos. The method is based on three components: 1) an architecture comprised of two LSTMs to model the sub-tasks of summarizing the past and inferring the future, 2) a Sequence Completion Pre-Training technique which encourages the LSTMs to focus on the different sub-tasks, and 3) a Modality ATTention (MATT) mechanism to efficiently fuse multi-modal predictions performed by processing RGB frames, optical flow fields and object-based features. The proposed approach is validated on EPIC-Kitchens, EGTEA Gaze+ and ActivityNet. The experiments show that the proposed architecture is state-of-the-art in the domain of egocentric videos, achieving top performances in the 2019 EPIC-Kitchens egocentric action anticipation challenge. The approach also achieves competitive performance on ActivityNet with respect to methods not based on unsupervised pre-training and generalizes to the tasks of early action recognition and action recognition. To encourage research on this challenging topic, we made our code, trained models, and pre-extracted features available at our web page: http://iplab.dmi.unict.it/rulstm.
Learning programs by learning from failures
Venue: arXiv cs.AI 2020
Year: 2020

We describe an inductive logic programming (ILP) approach called learning from failures. In this approach, an ILP system (the learner) decomposes the learning problem into three separate stages: generate, test, and constrain. In the generate stage, the learner generates a hypothesis (a logic program) that satisfies a set of hypothesis constraints (constraints on the syntactic form of hypotheses). In the test stage, the learner tests the hypothesis against training examples. A hypothesis fails when it does not entail all the positive examples or entails a negative example. If a hypothesis fails, then, in the constrain stage, the learner learns constraints from the failed hypothesis to prune the hypothesis space, i.e. to constrain subsequent hypothesis generation. For instance, if a hypothesis is too general (entails a negative example), the constraints prune generalisations of the hypothesis. If a hypothesis is too specific (does not entail all the positive examples), the constraints prune specialisations of the hypothesis. This loop repeats until either (i) the learner finds a hypothesis that entails all the positive and none of the negative examples, or (ii) there are no more hypotheses to test. We introduce Popper, an ILP system that implements this approach by combining answer set programming and Prolog. Popper supports infinite problem domains, reasoning about lists and numbers, learning textually minimal programs, and learning recursive programs. Our experimental results on three domains (toy game problems, robot strategies, and list transformations) show that (i) constraints drastically improve learning performance, and (ii) Popper can outperform existing ILP systems, both in terms of predictive accuracies and learning times.
Generalized Planning With Deep Reinforcement Learning
Venue: arXiv cs.AI 2020
Year: 2020

A hallmark of intelligence is the ability to deduce general principles from examples, which are correct beyond the range of those observed. Generalized Planning deals with finding such principles for a class of planning problems, so that principles discovered using small instances of a domain can be used to solve much larger instances of the same domain. In this work we study the use of Deep Reinforcement Learning and Graph Neural Networks to learn such generalized policies and demonstrate that they can generalize to instances that are orders of magnitude larger than those they were trained on.
Adversarial Training against Location-Optimized Adversarial Patches
Venue: arXiv cs.CV 2020
Year: 2020

Deep neural networks have been shown to be susceptible to adversarial examples -- small, imperceptible changes constructed to cause mis-classification in otherwise highly accurate image classifiers. As a practical alternative, recent work proposed so-called adversarial patches: clearly visible, but adversarially crafted rectangular patches in images. These patches can easily be printed and applied in the physical world. While defenses against imperceptible adversarial examples have been studied extensively, robustness against adversarial patches is poorly understood. In this work, we first devise a practical approach to obtain adversarial patches while actively optimizing their location within the image. Then, we apply adversarial training on these location-optimized adversarial patches and demonstrate significantly improved robustness on CIFAR10 and GTSRB. Additionally, in contrast to adversarial training on imperceptible adversarial examples, our adversarial patch training does not reduce accuracy.
Many visits TSP revisited
Venue: arXiv cs.DS 2020
Year: 2020

We study the Many Visits TSP problem, where given a number $k(v)$ for each of $n$ cities and pairwise (possibly asymmetric) integer distances, one has to find an optimal tour that visits each city $v$ exactly $k(v)$ times. The currently fastest algorithm is due to Berger, Kozma, Mnich and Vincze [SODA 2019, TALG 2020] and runs in time and space $\mathcal{O}^*(5^n)$. They also show a polynomial space algorithm running in time $\mathcal{O}^*(16^{n+o(n)})$. In this work, we show three main results: (i) A randomized polynomial space algorithm in time $\mathcal{O}^*(2^nD)$, where $D$ is the maximum distance between two cities. By using standard methods, this results in $(1+ε)$-approximation in time $\mathcal{O}^*(2^nε^{-1})$. Improving the constant $2$ in these results would be a major breakthrough, as it would result in improving the $\mathcal{O}^*(2^n)$-time algorithm for Directed Hamiltonian Cycle, which is a 50 years old open problem. (ii) A tight analysis of Berger et al.'s exponential space algorithm, resulting in $\mathcal{O}^*(4^n)$ running time bound. (iii) A new polynomial space algorithm, running in time $\mathcal{O}(7.88^n)$.
Coverage and Deployment Analysis of Narrowband Internet of Things in the Wild
Venue: arXiv cs.NI 2020
Year: 2020

Narrowband Internet of Things (NB-IoT) is gaining momentum as a promising technology for massive Machine Type Communication (mMTC). Given that its deployment is rapidly progressing worldwide, measurement campaigns and performance analyses are needed to better understand the system and move toward its enhancement. With this aim, this paper presents a large scale measurement campaign and empirical analysis of NB-IoT on operational networks, and discloses valuable insights in terms of deployment strategies and radio coverage performance. The reported results also serve as examples showing the potential usage of the collected dataset, which we make open-source along with a lightweight data visualization platform.
Fast Dynamic Cuts, Distances and Effective Resistances via Vertex Sparsifiers
Venue: arXiv cs.DS 2020
Year: 2020

We present a general framework of designing efficient dynamic approximate algorithms for optimization on undirected graphs. In particular, we develop a technique that, given any problem that admits a certain notion of vertex sparsifiers, gives data structures that maintain approximate solutions in sub-linear update and query time. We illustrate the applicability of our paradigm to the following problems. (1) A fully-dynamic algorithm that approximates all-pair maximum-flows/minimum-cuts up to a nearly logarithmic factor in $\tilde{O}(n^{2/3})$ amortized time against an oblivious adversary, and $\tilde{O}(m^{3/4})$ time against an adaptive adversary. (2) An incremental data structure that maintains $O(1)$-approximate shortest path in $n^{o(1)}$ time per operation, as well as fully dynamic approximate all-pair shortest path and transshipment in $\tilde{O}(n^{2/3+o(1)})$ amortized time per operation. (3) A fully-dynamic algorithm that approximates all-pair effective resistance up to an $(1+ε)$ factor in $\tilde{O}(n^{2/3+o(1)} ε^{-O(1)})$ amortized update time per operation. The key tool behind result (1) is the dynamic maintenance of an algorithmic construction due to Madry [FOCS' 10], which partitions a graph into a collection of simpler graph structures (known as j-trees) and approximately captures the cut-flow and metric structure of the graph. The $O(1)$-approximation guarantee of (2) is by adapting the distance oracles by [Thorup-Zwick JACM `05]. Result (3) is obtained by invoking the random-walk based spectral vertex sparsifier by [Durfee et al. STOC `19] in a hierarchical manner, while carefully keeping track of the recourse among levels in the hierarchy.
Information-Theoretic Security of MIMO Networks under $κ$-$μ$ Shadowed Fading Channels
Venue: arXiv cs.IT 2020
Year: 2020

This paper investigates the impact of realistic propagation conditions on the achievable secrecy performance of multiple-input multiple-output systems in the presence of an eavesdropper. Specifically, we concentrate on the $κ$-$μ$ shadowed fading model because its physical underpinnings capture a wide range of propagation conditions, while, at the same time, it allows for much better tractability than other state-of-the-art fading models. By considering transmit antenna selection and maximal ratio combining reception at the legitimate and eavesdropper's receiver sides, we study two relevant scenarios $(i)$ the transmitter does not know the eavesdropper's channel state information (CSI), and $(ii)$ the transmitter has knowledge of the CSI of the eavesdropper link. For this purpose, we first obtain novel and tractable expressions for the statistics of the maximum of independent and identically distributed (i.i.d.) variates related to the legitimate path. Based on these results, we derive novel closed-form expressions for the secrecy outage probability (SOP) and the average secrecy capacity (ASC) to assess the secrecy performance in passive and active eavesdropping scenarios, respectively. Moreover, we develop analytical asymptotic expressions of the SOP and ASC at the high signal-to-noise ratio regime. In all instances, secrecy performance metrics are characterized in closed-form, without requiring the evaluation of Meijer or Fox functions. Some useful insights on how the different propagation conditions and the number of antennas impact the secrecy performance are also provided.
Performance of RPL in Healthcare Wireless Sensor Network
Venue: arXiv cs.NI 2020
Year: 2020

The new advances of the Internet of Things (IoT) technology can be utilized to promote service delivery in several real-life applications such as healthcare systems. The Routing Protocol for Low Power and Loss Network (RPL) is a routing protocol designed to serve as a proper routing protocol for packets in Wireless Sensor Networks (WSN). Among the most prominent issues exist in the RPL protocol are packet loss within the WSN and sensors power consumption especially in healthcare WSNs. Multiple Objective Functions (OF) in RPL intended to find the routes from source nodes to a destination node. This paper presents an evaluation to discover which OF is more efficient for a WSN in a healthcare scenario where the Packet Delivery Ratio (PDR) of WSN and the sensors' power consumption are prominent concerns. Expected transmission Count (ETX) and Objective Function Zero (OF0) of RPL were examined in various network densities and network topologies such as the grid and random topology. The simulation outcomes revealed that the OF0 is more efficient regarding the PDR and power consumption compared to the ETX in random
Wireless Technologies for Agricultural Monitoring using Internet of Things Devices with Energy Harvesting Capabilities
Venue: arXiv cs.NI 2020
Year: 2020

Technological advances in the Internet of Things (IoT) have lead the way for technology to be used in ways that were never possible before. Through the development of devices with low-power radios, Wireless Sensor Networks (WSN) can be configured for almost any type of application. Agricultural has been one example where IoT and WSN have been able to increase productivity, efficiency, and output yield. Systems that previously required manual operation can be easily replaced with sensors and actuators to automate the process such as irrigation and disease management. Powering these devices is a concern as batteries are often required due to devices being located where electricity is not readily available. In this paper, a comparison is performed between three wireless technologies: IEEE 802.15.4 (Zigbee), Long Range Wireless Area Network (LoRaWAN), and IEEE 802.11g (WiFi 2.4~GHz) for agricultural monitoring with energy harvesting capabilities. According to experimental results, LoRaWAN is the optimal technology to use in an agricultural monitoring system where power consumption and network lifetime are a priority. The experimental results can be used for the selection of wireless technology for agricultural monitoring following application requirements.
The Pace and Pulse of the Fight against Coronavirus across the US, A Google Trends Approach
Venue: arXiv cs.CY 2020
Year: 2020

The coronavirus pandemic is impacting our lives at unprecedented speed and scale - including how we eat and work, what we worry about, how much we move, and our ability to earn. Google Trends can be used as a proxy for what people are thinking, needing, and planning. We use it to provide both insights into, and potential indicators of, important changes in information-seeking patterns during pandemics like COVID-19. Key questions we address are: (1) What is the relationship between the coronavirus outbreak and internet searches related to healthcare seeking, government support programs, media sources of different ideologies, planning around social activities, travel, and food, and new coronavirus-specific behaviors and concerns?; (2) How does the popularity of search terms differ across states and regions and can we explain these differences?; (3) Can we find distinct, tangible search patterns across states suggestive of policy gaps to inform pandemic response? (4) Does Google Trends data correlate with and potentially precede real-life events? We suggest strategic shifts for policy makers to improve the precision and effectiveness of non-pharmaceutical interventions (NPIs) and recommend the development of a real-time dashboard as a decision-making tool. Methods used include trend analysis of US search data; geographic analyses of the differences in search popularity across US states during March 1st to April 15th, 2020; and Principal Component Analyses (PCA) to extract search patterns across states.
Enabling Continuous Operations for UAVs with an Autonomous Service Network Infrastructure
Venue: arXiv cs.NI 2020
Year: 2020

One of the major restrictions on the practical applications of unmanned aerial vehicles (UAV) is their incomplete self-sufficiency, which makes continuous operations infeasible without human oversights. The more oversight UAVs require, the less likely they are going to be commercially advantageous when compared to their alternatives. As an autonomous system, how much human interaction is needed to function is one of the best indicators evaluating the limitations and inefficiencies of the UAVs. Popular UAV related research areas, such as path planning and computer vision, have enabled substantial advances in the ability of drones to act on their own. This research is dedicated to in-flight operations, in which there is not much reported effort to tackle the problem from the aspect of the supportive infrastructure. In this paper, an Autonomous Service network infrastructure (AutoServe) is proposed. Aiming at increasing the future autonomy of UAVs, the AutoServe system includes a service-oriented landing platform and a customized communication protocol. This supportive AutoServe infrastructure will autonomize many tasks currently done manually by human operators, such as battery replacement. A proof-of-concept prototype has been built and the simulation experimental study validated the design.
Phonetic and Visual Priors for Decipherment of Informal Romanization
Venue: arXiv cs.CL 2020
Year: 2020

Informal romanization is an idiosyncratic process used by humans in informal digital communication to encode non-Latin script languages into Latin character sets found on common keyboards. Character substitution choices differ between users but have been shown to be governed by the same main principles observed across a variety of languages---namely, character pairs are often associated through phonetic or visual similarity. We propose a noisy-channel WFST cascade model for deciphering the original non-Latin script from observed romanized text in an unsupervised fashion. We train our model directly on romanized data from two languages: Egyptian Arabic and Russian. We demonstrate that adding inductive bias through phonetic and visual priors on character mappings substantially improves the model's performance on both languages, yielding results much closer to the supervised skyline. Finally, we introduce a new dataset of romanized Russian, collected from a Russian social network website and partially annotated for our experiments.
Compact Device Models for FinFET and Beyond
Venue: arXiv cs.ET 2020
Year: 2020

Compact device models play a significant role in connecting device technology and circuit design. BSIM-CMG and BSIM-IMG are industry standard compact models suited for the FinFET and UTBB technologies, respectively. Its surface potential based modeling framework and symmetry preserving properties make them suitable for both analog/RF and digital design. In the era of artificial intelligence / deep learning, compact models further enhanced our ability to explore RRAM and other NVM-based neuromorphic circuits. We have demonstrated simulation of RRAM neuromorphic circuits with Verilog-A based compact model at NCKU. Further abstraction with macromodels is performed to enable larger scale machine learning simulation.
Unsupervised Pre-trained Models from Healthy ADLs Improve Parkinson's Disease Classification of Gait Patterns
Venue: arXiv cs.LG 2020
Year: 2020

Application and use of deep learning algorithms for different healthcare applications is gaining interest at a steady pace. However, use of such algorithms can prove to be challenging as they require large amounts of training data that capture different possible variations. This makes it difficult to use them in a clinical setting since in most health applications researchers often have to work with limited data. Less data can cause the deep learning model to over-fit. In this paper, we ask how can we use data from a different environment, different use-case, with widely differing data distributions. We exemplify this use case by using single-sensor accelerometer data from healthy subjects performing activities of daily living - ADLs (source dataset), to extract features relevant to multi-sensor accelerometer gait data (target dataset) for Parkinson's disease classification. We train the pre-trained model using the source dataset and use it as a feature extractor. We show that the features extracted for the target dataset can be used to train an effective classification model. Our pre-trained source model consists of a convolutional autoencoder, and the target classification model is a simple multi-layer perceptron model. We explore two different pre-trained source models, trained using different activity groups, and analyze the influence the choice of pre-trained model has over the task of Parkinson's disease classification.