Reviewerly/exHarmony huggingface dataset:
Associating Facial Expressions and Upper-Body Gestures with Learning Tasks for Enhancing Intelligent Tutoring Systems

Abstract Learning involves a substantial amount of cognitive, social and emotional states. Therefore, recognizing and understanding these states in the context of learning is key in designing informed interventions and addressing the needs of the individual student to provide personalized education. In this paper, we explore the automatic detection of learner’s nonverbal behaviors involving hand-over-face gestures, head and eye movements and emotions via facial expressions during learning. The proposed computer vision-based behavior monitoring method uses a low-cost webcam and can easily be integrated with modern tutoring technologies. We investigate these behaviors in-depth over time in a classroom session of 40 minutes involving reading and problem-solving exercises. The exercises in the sessions are divided into three categories: an easy, medium and difficult topic within the context of undergraduate computer science. We found that there is a significant increase in head and eye movements as time progresses, as well as with the increase of difficulty level. We demonstrated that there is a considerable occurrence of hand-over-face gestures (on average 21.35%) during the 40 minutes session and is unexplored in the education domain. We propose a novel deep learning approach for automatic detection of hand-over-face gestures in images with a classification accuracy of 86.87%. There is a prominent increase in hand-over-face gestures when the difficulty level of the given exercise increases. The hand-over-face gestures occur more frequently during problem-solving (easy 23.79%, medium 19.84% and difficult 30.46%) exercises in comparison to reading (easy 16.20%, medium 20.06% and difficult 20.18%).
ERA: A Data Set and Deep Learning Benchmark for Event Recognition in Aerial Videos [Software and Data Sets]

As a result of the increasing use of unmanned aerial vehicles (UAVs), large volumes of aerial videos have been produced. It is unrealistic for humans to screen such big data and understand the contents. Hence, methodological research on the automatic understanding of UAV videos is of paramount importance (Figure 1). In this article, we introduce a novel problem of event recognition in unconstrained aerial videos in the remote sensing community and present the large-scale, human-annotated Event Recognition in Aerial Videos (ERA) data set, consisting of 2,864 videos, each with a label from 25 different classes corresponding to an event unfolding for five seconds. All these videos are collected from YouTube. The ERA data set is designed to have significant intraclass variation and interclass similarity and captures dynamic events in various circumstances and at dramatically various scales. Moreover, to offer a benchmark for this task, we extensively validate existing deep networks. We expect that the ERA data set will facilitate further progress in automatic aerial video comprehension. The data set and trained models can be downloaded from https://lcmou.github.io/ERA_Dataset/.
ICDAR 2019 Competition on Harvesting Raw Tables from Infographics (CHART-Infographics)

This work summarizes the results of the first Competition on Harvesting Raw Tables from Infographics (ICDAR 2019 CHART-Infographics). The complex process of automatic chart recognition is divided into multiple tasks for the purpose of this competition, including Chart Image Classification (Task 1), Text Detection and Recognition (Task 2), Text Role Classification (Task 3), Axis Analysis (Task 4), Legend Analysis (Task 5), Plot Element Detection and Classification (Task 6.a), Data Extraction (Task 6.b), and End-to-End Data Extraction (Task 7). We provided a large synthetic training set and evaluated submitted systems using newly proposed metrics on both synthetic charts and manually-annotated real charts taken from scientific literature. A total of 8 groups registered for the competition out of which 5 submitted results for tasks 1-5. The results show that some tasks can be performed highly accurately on synthetic data, but all systems did not perform as well on real world charts. The data, annotation tools, and evaluation scripts have been publicly released for academic use.
Ensemble model output statistics as a probabilistic site-adaptation tool for satellite-derived and reanalysis solar irradiance

Suppose a location is within the spatial coverage of m gridded solar irradiance products, there is very little reason to rely on a single product, even if that product is known to be superior to its peers. In this paper, I discuss the ensemble performance of gridded irradiance estimates. First, I show the optimal convex combination of gridded irradiance estimates from different products almost always outperforms the best individual estimate under squared loss. Then, I extend the problem to the probability space and demonstrate how to construct predictive distributions for gridded irradiance estimates. Since the sample ensemble variances are often over- or under-dispersed, depending on the location, an ensemble model output statistics (EMOS) technique is used to correct such behaviors. The EMOS technique aims at minimizing the ignorance score, which is equivalent to maximizing the log-likelihood function of the underlying statistical model. In the language of solar engineers, EMOS is a probabilistic site-adaptation technique. At this point, this is the first work that (1) performs probabilistic site adaptation, (2) uses ensemble approaches for site adaptation, and (3) demonstrates formal probabilistic verification on site-adaptation problems.
Hash Access: Trustworthy Grant-Free IoT Access Enabled by Blockchain Radio Access Networks

The explosion of Internet of Things (IoT) applications makes the existing network structure insufficient because it is split by multiple non-cooperative operators, does not scale with the future development of IoT, and calls for novel networking concepts and protocols. The need for multi-operator IoT paradigms poses some unprecedented challenges, including trust considerations. To enable a multi-operator network without shared authentication, we propose a trustworthy grant-free IoT access protocol named Hash Access by leveraging a blockchain radio access network (B-RAN) architecture. B-RAN re-organizes multiple individual radio access networks into a joint multi-operator network based on a blockchain. B-RAN can establish trust among initially trustless operators and efficiently utilize resources across networks, matching and creating values for multiple participants. Furthermore, to mitigate the loss of network efficiency caused by selfish behavior among untrustworthy IoT devices, the proposed protocol can induce IoT devices to follow an access rule and control their data traffic. The proposed framework unifies the IoT access and the underlying B-RAN to reduce confirmation delays in traditional blockchains significantly. Numerical experiments illustrate the performance edge for the B-RAN-enabled Hash Access.
Full classification of permutation rational functions and complete rational functions of degree three over finite fields

Let q be a prime power, $$\mathbb {F}_q$$ be the finite field of order q and $$\mathbb {F}_q(x)$$ be the field of rational functions over $$\mathbb {F}_q$$. In this paper we classify and count all rational functions $$\varphi \in \mathbb {F}_q(x)$$ of degree 3 that induce a permutation of $$\mathbb {P}^1(\mathbb {F}_q)$$. As a consequence of our classification, we can show that there is no complete permutation rational function of degree 3 unless $$3\mid q$$ and $$\varphi $$ is a polynomial.
Active Learning of Reward Dynamics from Hierarchical Queries

Enabling robots to act according to human preferences across diverse environments is a crucial task, extensively studied by both roboticists and machine learning researchers. To achieve it, human preferences are often encoded by a reward function which the robot optimizes for. This reward function is generally static in the sense that it does not vary with time or the interactions. Unfortunately, such static reward functions do not always adequately capture human preferences, especially, in non-stationary environments: Human preferences change in response to the emergent behaviors of the other agents in the environment. In this work, we propose learning reward dynamics that can adapt in non-stationary environments with several interacting agents. We define reward dynamics as a tuple of reward functions, one for each mode of interaction, and mode-utility functions governing transitions between the modes. Reward dynamics thereby encodes not only different human preferences but also how the preferences change. Our contribution is in the way we adapt preference-based learning into a hierarchical approach that aims at learning not only reward functions but also how they evolve based on interactions. We derive a probabilistic observation model of how people will respond to the hierarchical queries. Our algorithm leverages this model to actively select hierarchical queries that will maximize the volume removed from a continuous hypothesis space of reward dynamics. We empirically demonstrate reward dynamics can match human preferences accurately.
A Fast and Accurate Fully Convolutional Network for End-to-End Handwritten Chinese Text Segmentation and Recognition

Handwritten Chinese Text Recognition (HCTR) is a challenging problem due to its high complexity. Previous methods based on over-segmentation, hidden Markov model (HMM) or long short-term memory recurrent neural network (LSTM-RNN) have achieved great success in recognition results. However, all of them, including over-segmentation based methods, are incompetent in accurate segmentation of single character. To solve this problem, we propose a fast and accurate fully convolutional network for end-to-end segmentation and recognition of handwritten Chinese text. Experiments on CASIA-HWDB datasets and ICDAR 2013 competition dataset show that our method achieves a competitive performance on recognition and produces great character segmentation results. Moreover, our model reaches a real-time speed of 70 fps, which is fast enough for various applications.
Explore Locally, Plan Globally: A Path Planning Framework for Autonomous Robotic Exploration in Subterranean Environments

This paper presents a path planning strategy for the autonomous exploration of subterranean environments. Tailored to the specific challenges and particularities of underground settings, and especially the fact that they often are extremely large in scale, tunnel-like, narrow and multibranched, the proposed method employs a bifurcated local- and global-planning design to enable exploration efficiency and path planning solution resourcefulness. The local planner builds on top of minimum-length random trees and efficiently identifies collision-free paths that optimize for exploration within a local subspace, while simultaneously ensuring enhanced obstacle clearance and thus safety. Accounting for the robot endurance limitations and the possibility that the local planner reaches a dead-end (e.g. a mine heading), the global planner utilizes an incrementally built graph to search within the full range of explored space and is engaged when the robot should be repositioned towards a frontier of the exploration space or when a return-to-home path must be derived. The proposed approach is field evaluated in a set of deployments in an exploratory underground mine drift in Northern Nevada.
Personalized Stride-Length Estimation Based on Active Online Learning

The ability to accurately estimate a user's stride length plays a great important role in various applications. For a new target pedestrian or device, their heterogeneity dramatically reduces the performance of the current stride-length estimation (SLE) methods. To address the issue of heterogeneity, in this article, we propose an SLE method based on a long short-term memory (LSTM) network and denoising autoencoders (DAEs). The LSTM network is used to mine temporal dependencies and extract significant eigenvectors from the corrupted inertial sensor observations. Then, DAEs are adopted to automatically eliminate the inherent noise in eigenvectors and obtain denoised eigenvectors. Finally, a regression module maps the denoised eigenvectors to the resulting stride length. To mitigate the heterogeneity, we propose an unperceived model updating framework based on active online learning to establish a personalized model for a given target pedestrian or device. The proposed framework utilizes a magnetism-aided map-matching approach to automatically generate personalized training data and utilizes online learning technologies to evolve the stride-length model. The extensive experimental results demonstrate that the proposed method outperforms other state-of-the-art algorithms and achieves a promising accuracy with a stride-length error rate of 4.59% at a confidence level of 80%.
QoS-Aware Personalized Privacy With Multipath TCP for Industrial IoT: Analysis and Design

With the ensuing surge in data communication volume and the growing need for privacy protection, limiting centralized data collection to the minimum required for specific tasks has been mandatory in industries. This is now guided by the modern privacy legislation, namely, the General Data Protection Regulation and the California Consumer Protection Act. Privacy leakage has become increasingly serious because of massive volume and a variety of data transmission and Quality-of-Service (QoS) requirements in the Industrial Internet-of-Things (IIoT) networks. Although differential privacy is the core privacy protection paradigm, most of its extensions assume all parties share the same level of privacy requirements, which cannot meet varying needs and QoS of IIoT devices in practice. In addition, with multiple paths access to the cloud server (often operated by the trusted third party in IIoT) for higher reliability and performance, satisfying both the privacy and QoS is nontrivial during the data transmission. The usual transmission over both the cellular and WiFi interfaces simultaneously for continuous connectivity among devices, edge networks, and the server is crucial. As a result, we observe that IIoT data privacy is highly vulnerable to collusion attacks. Motivated by this observation, we develop a detailed QoS modeling for multipath TCP over IIoT and propose a QoS-aware personalized privacy protection model. Our model works in two different layers: one at the cloud server and another at the network edges (access points/base station). The aim is not only to balance the load but also to achieve the required QoS and optimize the tradeoff between privacy protection and efficiency. The extensive experimental results based on the real-world data sets illustrate the superiority of the proposed model in terms of privacy protection and efficiency.
Face Recognition Using Faster R-CNN with Inception-V2 Architecture for CCTV Camera

Detection and prevention of criminal incidents using CCTV are currently increasing trend, for example, car and motorcycle parking lot. However, not continuous people monitoring and careless of events produce useless CCTV function for the prevention of criminal incidents. In this paper, face recognition is used for the recognition of vehicle owners in parking lots that are CCTV installed. The Faster-RCNN method is used for face detection and also for face recognition. Inception V2 architecture is utilized due to has a high accuracy among Convolutional Neural Network architecture. The best learning rate and epoch parameters for the Faster R-CNN model are optimized to improve face recognition on CCTV. In this research, the dataset consists of 6 people images with 50 faces images for each people, which used as training data, testing data, and validation data.
A Security Transmission and Storage Solution about Sensing Image for Blockchain in the Internet of Things

With the rapid development of the Internet of Things (IoT), the number of IoT devices has increased exponentially. Therefore, we have put forward higher security requirements for the management, transmission, and storage of massive IoT data. However, during the transmission process of IoT data, security issues, such as data theft and forgery, are prone to occur. In addition, most existing data storage solutions are centralized, i.e., data are stored and maintained by a centralized server. Once the server is maliciously attacked, the security of IoT data will be greatly threatened. In view of the above-mentioned security issues, a security transmission and storage solution is proposed about sensing image for blockchain in the IoT. Firstly, this solution intelligently senses user image information, and divides these sensed data into intelligent blocks. Secondly, different blocks of data are encrypted and transmitted securely through intelligent encryption algorithms. Finally, signature verification and storage are performed through an intelligent verification algorithm. Compared with the traditional IoT data transmission and centralized storage solution, our solution combines the IoT with the blockchain, making use of the advantages of blockchain decentralization, high reliability, and low cost to transfer and store users image information securely. Security analysis proves that the solution can resist theft attacks and ensure the security of user image information during transmission and storage.
Flexible spatial location-based PVO predictor for high-fidelity reversible data hiding

An advanced predictor offers the most effective way for performance enhancement of prediction-error expansion (PEE) based reversible data hiding. Unlike conventional predictors which exploit the correlation among adjacent pixels, pixel-value-ordering (PVO) exploits the one between the largest/smallest two pixels within block regardless of where they are located. Later, the spatial location received the attention from Peng et al. and led to better expansion bins selection. In this paper, a flexible spatial location (FSL) strategy is proposed to optimize the utilization of spatial correlation in PVO prediction. With FSL, pixels within block are no longer collected in a fixed way (e.g., from top to bottom, left to right). Specifically, eight modes of defining spatial location are designed for pixel collection. After pixel sorting, the inverse number of the location sequence is used to evaluate all modes and determine the best one. By determining the optimal mode for each block, the prediction becomes adaptive and thus a prediction-error histogram with higher expansion bin will be obtained. Numerous PVO-based schemes will benefit from FSL. In this paper, how FSL is applied to conventional PEE and high-dimensional PEE is also introduced. Experimental results demonstrate that FSL-PVO is of great significance to better capacity-distortion trade-off.
Restricted modes: Social media, content classification and LGBTQ sexual citizenship

In the context of recent controversies surrounding the censorship of lesbian, gay, bisexual, transgender and queer online content, specifically on YouTube and Tumblr, we interrogate the relationship between normative understandings of sexual citizenship and the content classification regimes. We argue that these content classification systems and the platforms’ responses to public criticism both operate as norm-producing technologies, in which the complexities of sexuality and desire are obscured in order to cultivate notions of a ‘good’ lesbian, gay, bisexual, transgender or queer sexual citizen. However, despite normative work of classification seeking to distinguish between sexuality and sex, we argue that the high-profile failures of these classification systems create the conditions for users to draw attention to, rather than firm, these messy boundaries.
EvilScout: Detection and Mitigation of Evil Twin Attack in SDN Enabled WiFi

Spoofing the identity of a WiFi access point (AP) is trivial. Consequently, an adversary can impersonate the legitimate AP (LAP) by mimicking its network name (SSID) and MAC address (BSSID). This fake AP is called the evil twin. An evil twin can perform multiple attacks such as man-in-the-middle (MITM) attack between the LAP and a wireless client as well as service blocking of LAP. Existing solutions rely on the collection and calculation of information with the AP and/or client for finding evidence of evil twins in the WiFi network. Some of them require additional hardware to acquire further information that cannot be provided by the AP/client. In this paper, we propose “EvilScout,” an evil twin detection and mitigation framework that utilizes the information of the IP-prefix distribution by the LAP. EvilScout exploits the SDN potential for detection of an evil twin without the need of any additional hardware or modifications at the AP or client. Additionally, the information that becomes available at the SDN controller enables simplified and more accurate evil twin detection. This paper presents the implementation of EvilScout over a real SDN WiFi testbed with an actual evil twin. We verify the successful detection of the evil twin with high accuracy and low processing cost at the SDN WiFi. We perform a rigorous analysis of the evil twin in different WiFi setups and discover a new “AP Service Blocking” attack by the evil twin adversary in the WPA2 protected WiFi for the first time.
Automated digital cause-and-effect diagrams to assist causal analysis in problem-solving: a data-driven approach

Causal analysis is an integral part of product quality problem-solving (QPS). Quality management within the manufacturing industry has generated a considerable amount of QPS data; while this implies a historical and extensive body of QPS experience, these valuable empirical data are not being fully utilised. Therefore, the current study proposes a method by which to mine know-why from historical empirical data, and it develops an approach for constructing digital cause-and-effect diagrams (CEDs). The K-means algorithm is first adopted to cluster the problems and causes. The random forest classifier is then selected to classify cause text into the main cause categories, which manifest as 'rib branches' in the CED. Based on the clustering and classification results, we obtain an abstract cause-and-effect diagram (ACED) and a detailed cause-and-effect diagram (DCED). We use the quality data of an automotive company to validate the method, and we additionally undertake a pilot run of the Fishbone Next system to demonstrate how users can obtain these two CEDs to support causal analysis in QPS. The results show that the proposed approach efficiently constructs a digital CED and thus provides quality management problem-solvers with decision support to derive the potential causes of problems, thereby improving the efficiency and effectiveness of their causal analysis initiatives.