Reviewerly/exHarmony huggingface dataset:
A game-based approximate verification of deep neural networks with provable guarantees

Despite the improved accuracy of deep neural networks, the discovery of adversarial examples has raised serious safety concerns. In this paper, we study two variants of pointwise robustness, the maximum safe radius problem, which for a given input sample computes the minimum distance to an adversarial example, and the feature robustness problem, which aims to quantify the robustness of individual features to adversarial perturbations. We demonstrate that, under the assumption of Lipschitz continuity, both problems can be approximated using finite optimisation by discretising the input space, and the approximation has provable guarantees, i.e., the error is bounded. We then show that the resulting optimisation problems can be reduced to the solution of two-player turn-based games, where the first player selects features and the second perturbs the image within the feature. While the second player aims to minimise the distance to an adversarial example, depending on the optimisation objective the first player can be cooperative or competitive. We employ an anytime approach to solve the games, in the sense of approximating the value of a game by monotonically improving its upper and lower bounds. The Monte Carlo tree search algorithm is applied to compute upper bounds for both games, and the Admissible A⁎ and the Alpha-Beta Pruning algorithms are, respectively, used to compute lower bounds for the maximum safety radius and feature robustness games. When working on the upper bound of the maximum safe radius problem, our tool demonstrates competitive performance against existing adversarial example crafting algorithms. Furthermore, we show how our framework can be deployed to evaluate pointwise robustness of neural networks in safety-critical applications such as traffic sign recognition in self-driving cars.
Emerging topic detection in twitter stream based on high utility pattern mining

Among internet and smart device applications, Twitter has become a leading social media platform, disseminating online events occurring in the world on a real-time basis. Many studies have been conducted to identify valuable information on Twitter. Recently, Frequent Pattern Mining has been applied for topic detection on Twitter. In Frequent Pattern Mining, a topic is considered to be a group of words that appear simultaneously, however, the method only considers the frequency of words, and their utility for topic detection is not considered in the process of pattern generation. In this paper, we propose a method to detect emerging topics on Twitter based on High Utility Pattern Mining (HUPM), which takes frequency and utility into account at the same time. For a chunk of tweets by time-based windowing on the Twitter stream, we define the utility of words based on the growth rate in frequency and find groups of words with high frequency and high utility by HUPM. For post-processing to extract actual topic patterns from candidate topic patterns generated by HUPM, an efficient data structure called Topic-tree (TP-Tree) is also proposed. Experimental results demonstrated the effectiveness of the proposed method, which showed superior performance and shorter running time than other tested topic detection methods. In particular, the proposed method showed a 5% higher topic recall than the other compared methods for the three datasets used.
2D-LBP: An Enhanced Local Binary Feature for Texture Image Classification

The local binary pattern (LBP) and its variants have shown the effectiveness in texture images classification, face recognition, and other applications. However, most of these LBP methods only focus on the histogram of LBP patterns and ignore the spatial contextual information between LBP patterns. In this paper, we propose a 2D-LBP method which uses a sliding window to count the weighted occurrence number of the rotation invariant uniform LBP pattern pairs to obtain the spatial contextual information. The multi-resolution 2D-LBP features can also be obtained when the radius of 2D-LBP is changed. At last, a two-stage classifier which acts as an ensemble learning step is followed to achieve an accurate classification by combining the predictions on each 2D-LBP with single resolution. Theoretical proof shows that the proposed 2D-LBP is a general framework and can be integrated on other LBP variants to derive new feature extraction methods. Experimental results show that, the proposed method achieves 99.71%, 97.09%, 98.48%, and 49.00% classification accuracy on the public “Brodatz,” “CUReT,” “UIUC,” and “FMD” texture image databases, respectively. Compared with the original LBP and its variants, the proposed method obtains higher classification accuracy under different cases, and simultaneously owns shorter time complexity.
Modality-correlation-aware sparse representation for RGB-infrared object tracking

To intelligently analyze and understand video content, a key step is to accurately perceive the motion of the interested objects in videos. To this end, the task of object tracking, which aims to determine the position and status of the interested object in consecutive video frames, is very important, and has received great research interest in the last decade. Although numerous algorithms have been proposed for object tracking in RGB videos, most of them may fail to track the object when the information from the RGB video is not reliable (e.g. in dim environment or large illumination change). To address this issue, with the popularity of dual-camera systems for capturing RGB and infrared videos, this paper presents a feature representation and fusion model to combine the feature representation of the object in RGB and infrared modalities for object tracking. Specifically, this proposed model is able to (1) perform feature representation of objects in different modalities by employing the robustness of sparse representation, and (2) combine the representation by exploiting the modality correlation. Extensive experiments demonstrate the effectiveness of the proposed method.
Which Has Better Visual Quality: The Clear Blue Sky or a Blurry Animal?

Image content variation is a typical and challenging problem in no-reference image-quality assessment (NR-IQA). This work pays special attention to the impact of image content variation on NR-IQA methods. To better analyze this impact, we focus on blur-dominated distortions to exclude the impacts of distortion-type variations. We empirically show that current NR-IQA methods are inconsistent with human visual perception when predicting the relative quality of image pairs with different image contents. In view of deep semantic features of pretrained image classification neural networks always containing discriminative image content information, we put forward a new NR-IQA method based on semantic feature aggregation (SFA) to alleviate the impact of image content variation. Specifically, instead of resizing the image, we first crop multiple overlapping patches over the entire distorted image to avoid introducing geometric deformations. Then, according to an adaptive layer selection procedure, we extract deep semantic features by leveraging the power of a pretrained image classification model for its inherent content-aware property. After that, the local patch features are aggregated using several statistical structures. Finally, a linear regression model is trained for mapping the aggregated global features to image-quality scores. The proposed method, SFA, is compared with nine representative blur-specific NR-IQA methods, two general-purpose NR-IQA methods, and two extra full-reference IQA methods on Gaussian blur images (with and without Gaussian noise/JPEG compression) and realistic blur images from multiple databases, including LIVE, TID2008, TID2013, MLIVE1, MLIVE2, BID, and CLIVE. Experimental results show that SFA is superior to the state-of-the-art NR methods on all seven databases. It is also verified that deep semantic features play a crucial role in addressing image content variation, and this provides a new perspective for NR-IQA.
Adaptive-weighting discriminative regression for multi-view classification

Multi-view data represented by different features have been involved in many machine learning applications. Efficiently exploiting and preserving the correlative yet complementary information in multiple views remains challenging in multi-view learning. Comparing with existing methods that separately cope with each view, we propose a supervised multi-view feature learning framework to handle diverse views with a unified perception. Specifically, we fuse the multi-view data by mapping the concatenation of original features to a discriminative low-dimensional subspace, where the features from different views are adaptively assigned with the learned optimal weights. This strategy can simultaneously preserve the correlative and the complementary information, which is further enhanced to be more discriminative for subsequent classification. An efficient iterative algorithm is devised to optimize the formulated framework with closed-form solutions. Comprehensive evaluations with several state-of-the-art competitors demonstrate the efficiency and the superiority of the proposed method.
Robot-enabled support of daily activities in smart home environments

Smart environments offer valuable technologies for activity monitoring and health assessment. Here, we describe an integration of robots into smart environments to provide more interactive support of individuals with functional limitations. RAS, our Robot Activity Support system, partners smart environment sensing, object detection and mapping, and robot interaction to detect and assist with activity errors that may occur in everyday settings. We describe the components of the RAS system and demonstrate its use in a smart home testbed. To evaluate the usability of RAS, we also collected and analyzed feedback from participants who received assistance from RAS in a smart home setting as they performed routine activities.
Towards automatic fingerprinting of IoT devices in the cyberspace

Nowadays, the cyberspace consists of an increasing number of IoT devices, such as net-printers, webcams, and routers. Illuminating the nature of online devices would provide insights into detecting potentially vulnerable devices on the Internet. However, there is a lack of device discovery in large-scale due to the massive number of device models (i.e., types, vendors, and products). In this paper, we propose an efficient approach to generate fingerprints of IoT devices. We observe that device manufacturers have different network system implementations on their products. We explore features spaces of IoT devices in three network layers, including the network-layer, transport-layer, and application-layer. Utilizing the feature of network protocols, we generate IoT device fingerprints based on neural network algorithms. Furthermore, we implement the prototype system and conduct real experiments to validate the performance of device fingerprints. Results show that our classification can generate device class labels with a 94% precision and 95% recall. We use those device fingerprints to discover 15.3 million network-connected devices and analyze their distribution characteristics in cyberspace.
Automatic Feature Learning for Predicting Vulnerable Software Components

Code flaws or vulnerabilities are prevalent in software systems and can potentially cause a variety of problems including deadlock, hacking, information loss and system failure. A variety of approaches have been developed to try and detect the most likely locations of such code vulnerabilities in large code bases. Most of them rely on manually designing code features (e.g., complexity metrics or frequencies of code tokens) that represent the characteristics of the potentially problematic code to locate. However, all suffer from challenges in sufficiently capturing both semantic and syntactic representation of source code, an important capability for building accurate prediction models. In this paper, we describe a new approach, built upon the powerful deep learning Long Short Term Memory model, to automatically learn both semantic and syntactic features of code. Our evaluation on 18 Android applications and the Firefox application demonstrates that the prediction power obtained from our learned features is better than what is achieved by state of the art vulnerability prediction models, for both within-project prediction and cross-project prediction.
Technical aspects of blockchain and IoT

Blockchain technology is getting a growing attention from various organizations and researchers as it provides magical solutions to the problems associated with the classical centralized architecture. Blockchain, whether public or private, is a distributed ledger with the capability of maintaining the integrity of transactions by decentralizing the ledger among participating users. On the other hand, the Internet of Things (IoT) represents a revolution of the Internet which can connect nearly all environment devices over the Internet to share their data to create novel services and applications for improving our quality of life. Although the centralized IoT system provides countless benefits, it raises several challenges. Resolving these challenges can be done by integrating IoT with blockchain technology. To be prepared for the integration process, this chapter provides an overview of technical aspects of the blockchain and IoT. It started by reviewing blockchain technology and its main structure. Applications and challenges of the blockchain are also presented. This is followed by reviewing the IoT system by highlighting common architecture and essential characteristics. Various applications and challenges of the IoT system are also discussed.
360° Video Journalism: Experimental Study on the Effect of Immersion on News Experience and Distant Suffering

News producers are increasingly experimenting with news in virtual reality and 360°video, which is often presented as the ultimate form of immersive journalism as it provides viewers with a first-person experience of a news event.Audiences are deemed to be more involved with the presented story, raising assumptions that they may be more engaged with the event and feel more empathic towards the subject of the story, especially in the case of foreign news, as distance is virtually narrowed.This experimental study (n = 149) assesses whether 360°international disaster news leads to a higher sense of presence, higher enjoyment, higher subjective involvement towards the topic, and increased engagement with distant suffering towards the victim.Using an existing news item produced by the Belgian public broadcaster VRT on a Syrian oil worker, a between-subject experimental study was undertaken, comparing four 360°video conditions: single viewpoint, drag-anddrop 360°, 360°with a cardboard VR device, and 360°with a head-mounted VR device.A verbal, qualitative debrief allowed for a better understanding of the results, which show that 360°video journalism leads to a higher sense of presence and higher levels of enjoyment.However, no effect on distant suffering or subjective involvement is found.
Clustering ensemble based on sample's stability

The objective of clustering ensemble is to find the underlying structure of data based on a set of clustering results. It has been observed that the samples can change between clusters in different clustering results. This change shows that samples may have different contributions to the detection of the underlying structure. However, the existing clustering ensemble methods treat all sample equally. To tackle this deficiency, we introduce the stability of a sample to quantify its contribution and present a methodology to determine this stability. We propose two formulas accord with this methodology to calculate sample's stability. Then, we develop a clustering ensemble algorithm based on the sample's stability. With either formula, this algorithm divides a data set into two classes: cluster core and cluster halo. With the core and halo, the proposed algorithm then discovers a clear structure using the samples in the cluster core and assigns samples in the cluster halo to the clear structure gradually. The experiments on eight synthetic data sets illustrate how the proposed algorithm works. This algorithm statistically outperforms twelve state-of-the-art clustering ensemble algorithms on ten real data sets from UCI and six document data sets. The experimental analysis on the case of image segmentation shows that cluster cores discovered by the stability are rational.
Autonomous Vehicles: Autodriver Algorithm and Vehicle Dynamics

A given road can be expressed mathematically in a global (or world) coordinate frame. Following the road can be substituted by following the loci of its curvature center and turning at the right circle of curvature. Considering that a vehicle in motion is always in turn about an instantaneous rotation center relative to the ground, an autonomous vehicle capable of following a given path by coinciding the rotation center of vehicle at every moment on the curvature center of the road could be designed. The dynamic reactions of the vehicle influence its path of motion and make its rotation center to depart from the desired path of the curvature center of the road. In this study, the Autodriver algorithm control strategy to front-wheel-steering vehicles has been developed and a control loop is introduced to compensate the present errors generated by the differences of the desired locating on the road and the real position of the vehicle.
Cross-domain Recommendation Without Sharing User-relevant Data

Web systems that provide the same functionality usually share a certain amount of items. This makes it possible to combine data from different websites to improve recommendation quality, known as the cross-domain recommendation task. Despite many research efforts on this task, the main drawback is that they largely assume the data of different systems can be fully shared. Such an assumption is unrealistic - different systems are typically operated by different companies, and it may violate business privacy policy to directly share user behavior data since it is highly sensitive.
Detection of abnormal behaviour for dementia sufferers using Convolutional Neural Networks

In recent years, there is a rapid increase in the population of elderly people. However, elderly people may suffer from the consequences of cognitive decline, which is a mental health disorder that primarily affects cognitive abilities such as learning, memory, etc. As a result, the elderly people may get dependent on caregivers to complete daily life tasks. Detecting the early indicators of dementia before it gets worsen and warning the caregivers and medical doctors would be helpful for further diagnosis. In this paper, the problem of activity recognition and abnormal behaviour detection is investigated for elderly people with dementia. First of all, the paper presents a methodology for generating synthetic data reflecting on some behavioural difficulties of people with dementia given the difficulty of obtaining real-world data. Secondly, the paper explores Convolutional Neural Networks (CNNs) to model patterns in activity sequences and detect abnormal behaviour related to dementia. Activity recognition is considered as a sequence labelling problem, while abnormal behaviour is flagged based on the deviation from normal patterns. Moreover, the performance of CNNs is compared against the state-of-art methods such as Naïve Bayes (NB), Hidden Markov Models (HMMs), Hidden Semi-Markov Models (HSMM), Conditional Random Fields (CRFs). The results obtained indicate that CNNs are competitive with those state-of-art methods.
Efficient Circuit-Based PSI with Linear Communication

We present a new protocol for computing a circuit which implements the private set intersection functionality (PSI). Using circuits for this task is advantageous over the usage of specific protocols for PSI, since many applications of PSI do not need to compute the intersection itself but rather functions based on the items in the intersection. Our protocol is the first circuit-based PSI protocol to achieve linear communication complexity. It is also concretely more efficient than all previous circuit-based PSI protocols. For example, for sets of size $$2^{20}$$ it improves the communication of the recent work of Pinkas et al. (EUROCRYPT’18) by more than 10 times, and improves the run time by a factor of 2.8x in the LAN setting, and by a factor of 5.8x in the WAN setting. Our protocol is based on the usage of a protocol for computing oblivious programmable pseudo-random functions (OPPRF), and more specifically on our technique to amortize the cost of batching together multiple invocations of OPPRF.
Naturalness-Aware Deep No-Reference Image Quality Assessment

No-reference image quality assessment (NR-IQA) is a non-trivial task, because it is hard to find a pristine counterpart for an image in real applications, such as image selection, high quality image recommendation, etc. In recent years, deep learning-based NR-IQA methods emerged and achieved better performance than previous methods. In this paper, we present a novel deep neural networks-based multi-task learning approach for NR-IQA. Our proposed network is designed by a multi-task learning manner that consists of two tasks, namely, natural scene statistics (NSS) features prediction task and the quality score prediction task. NSS features prediction is an auxiliary task, which helps the quality score prediction task to learn better mapping between the input image and its quality score. The main contribution of this work is to integrate the NSS features prediction task to the deep learning-based image quality prediction task to improve the representation ability and generalization ability. To the best of our knowledge, it is the first attempt. We conduct the same database validation and cross database validation experiments on LIVE <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> , TID2013 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> , CSIQ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> , LIVE multiply distorted image quality database (LIVE MD) <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sup> , CID2013 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">5</sup> , and LIVE in the wild image quality challenge (LIVE challenge) <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">6</sup> databases to verify the superiority and generalization ability of the proposed method. Experimental results confirm the superior performance of our method on the same database validation; our method especially achieves 0.984 and 0.986 on the LIVE image quality assessment database in terms of the Pearson linear correlation coefficient (PLCC) and Spearman rank-order correlation coefficient (SROCC), respectively. Also, experimental results from cross database validation verify the strong generalization ability of our method. Specifically, our method gains significant improvement up to 21.8% on unseen distortion types.