Reviewerly/exHarmony huggingface dataset:
Joint Inference of States, Robot Knowledge, and Human (False-)Beliefs

Aiming to understand how human (false-)belief- a core socio-cognitive ability-would affect human interactions with robots, this paper proposes to adopt a graphical model to unify the representation of object states, robot knowledge, and human (false-)beliefs. Specifically, a parse graph (pg) is learned from a single-view spatiotemporal parsing by aggregating various object states along the time; such a learned representation is accumulated as the robot's knowledge. An inference algorithm is derived to fuse individual pg from all robots across multi-views into a joint pg, which affords more effective reasoning and inference capability to overcome the errors originated from a single view. In the experiments, through the joint inference over pgs, the system correctly recognizes human (false-)belief in various settings and achieves better cross-view accuracy on a challenging small object tracking dataset.
An Empirical Approach to Modeling User-System Interaction Conflicts in Smart Homes

Conflict is one of the important factors affecting user satisfaction and trust in smart environments, yet conflict modeling in mixed initiative smart environments has not been sufficiently explored. Most of the existing literature on conflict in smart homes are centered on conflicts between users. Although research has shown that about 75% of conflicts are between users and system [1], only a few studies have considered user-system conflicts in smart homes. The aim of this article is to empirically propose both a definition and a run-time detection method for conflicts between users and smart home systems. Our empirical study is based on conflict sample scenarios collected from 163 users. Using clustering on these scenarios, we form an empirical definition of user-system conflict in smart homes. We also propose two functions that characterize each class of the collected scenarios, and we detect conflicts from this characterization. Our conflict detection model could help users achieve a more satisfactory experience in smart homes. Moreover, the model can offer benefits for system developers to design and deploy more reliable smart homes.
Pedestrian as Points: An Improved Anchor-Free Method for Center-Based Pedestrian Detection

Although excessive proposals using traditional sliding-window methods or prevailing anchor-based techniques have been proposed to deal with deep learning-based pedestrian detection, it is still a promising yet challenging problem. In this paper, we propose a precise, flexible and thoroughly anchor-free, as well as proposal-free framework named Pedestrian-as-Points Network (PP-Net) for pedestrian detection. Specifically, we model a pedestrian as a single point, i.e., the center point of the instance, and predict the pedestrian scale at each detected center point. In order to achieve higher accuracy, we build a pyramid-like structure based on the backbone as a feature extractor to aggregate multi-level information. In addition, we construct a deep guidance module (DGM) at the top of the backbone, so that the higher-level information can be captured in the process of building a feature pyramid network (FPN) to avoid the dilution of high-level information on the top-down pathway. We further design a feature fusion unit (FFU) to fuse the fine-level features well with the coarse-level semantic information from the top-down pathway. With the only post-processing non-maximum suppression (NMS), we achieve better performance than many state-of-the-arts methods on the challenging pedestrian detection datasets.
Dual Grid Net: Hand Mesh Vertex Regression from Single Depth Maps

We aim to recover the dense 3D surface of the hand from depth maps and propose a network that can predict mesh vertices, transformation matrices for every joint and joint coordinates in a single forward pass. Use fully convolutional architectures, we first map depth image features to the mesh grid and then regress the mesh coordinates into real world 3D coordinates. The final mesh is found by sampling from the mesh grid refit in closed-form based on an articulated template mesh. When trained with supervision from sparse key-points, our accuracy is comparable with state-of-the-art on the NYU dataset for key point localization, all while recovering mesh vertices and dense correspondences. Under multi-view settings for training, our framework can also learn through self-supervision by minimizing a set of data-fitting terms and kinematic priors. Our approach is competitive with strongly supervised methods and showcases the potential for self-supervision in dense mesh estimation.
AUC Estimation and Concept Drift Detection for Imbalanced Data Streams with Multiple Classes

Online class imbalance learning deals with data streams having very skewed class distributions.When learning from data streams, concept drift is one of the major challenges that deteriorate the classification performance.Although several approaches have been recently proposed to overcome concept drift in imbalanced data, they are all limited to two-class cases.Multi-class imbalance imposes additional challenges in concept drift detection and performance evaluation, such as a more severe imbalanced distribution and the limited choice of performance measures.This paper extends AUC for evaluating classifiers on multi-class imbalanced data in online learning scenarios.The proposed metrics, PMAUC, WAUC and EWAUC, are studied through comprehensive experiments, focusing on their characteristics on time-changing data streams and whether and how they can be used to detect concept drift.The AUC-based metrics show effectiveness in detecting concept drift in a variety of artificial data streams and a real-world data application with multiple classes.In particular, EWAUC is shown to be both effective and efficient.
Securing unsafe rust programs with XRust

Rust is a promising systems programming language that embraces both high-level memory safety and low-level resource manipulation. However, the dark side of Rust, unsafe Rust, leaves a large security hole as it bypasses the Rust type system in order to support low-level operations. Recently, several real-world memory corruption vulnerabilities have been discovered in Rust's standard libraries.
First Steps Towards a Runtime Analysis When Starting with a Good Solution

The mathematical runtime analysis of evolutionary algorithms traditionally regards the time an algorithm needs to find a solution of a certain quality when initialized with a random population. In practical applications it may be possible to guess solutions that are better than random ones. We start a mathematical runtime analysis for such situations. We observe that different algorithms profit to a very different degree from a better initialization. We also show that the optimal parameterization of the algorithm can depend strongly on the quality of the initial solutions. To overcome this difficulty, self-adjusting and randomized heavy-tailed parameter choices can be profitable. Finally, we observe a larger gap between the performance of the best evolutionary algorithm we found and the corresponding black-box complexity. This could suggest that evolutionary algorithms better exploiting good initial solutions are still to be found. These first findings stem from analyzing the performance of the $$(1+1)$$ evolutionary algorithm and the static, self-adjusting, and heavy-tailed $$(1 + (\lambda ,\lambda ))$$ GA on the OneMax benchmark, but we are optimistic that the question how to profit from good initial solutions is interesting beyond these first examples.
A 3D-Deep-Learning-based Augmented Reality Calibration Method for Robotic Environments using Depth Sensor Data

Augmented Reality and mobile robots are gaining increased attention within industries due to the high potential to make processes cost and time efficient. To facilitate augmented reality, a calibration between the Augmented Reality device and the environment is necessary. This is a challenge when dealing with mobile robots due to the mobility of all entities making the environment dynamic. On this account, we propose a novel approach to calibrate Augmented Reality devices using 3D depth sensor data. We use the depth camera of a Head Mounted Augmented Reality Device, the Microsoft Hololens, for deep learning-based calibration. Therefore, we modified a neural network based on the recently published VoteNet architecture which works directly on raw point cloud input observed by the Hololens. We achieve satisfying results and eliminate external tools like markers, thus enabling a more intuitive and flexible work flow for Augmented Reality integration. The results are adaptable to work with all depth cameras and are promising for further research. Furthermore, we introduce an open source 3D point cloud labeling tool, which is to our knowledge the first open source tool for labeling raw point cloud data.
Software for Automatic Estimating Security Settings of Social Media Accounts

Today, many users do not use the security settings provided by social media and are unaware of the risks that come with sharing too much personal information. In this research, security management on social media is considered. The paper discusses and investigates the awareness of security settings that impact social media users. The software for automatic estimating security settings of social media accounts is developed. The developed software allows estimate the security settings of Facebook and Instagram social media accounts and formulates recommendations for their improvement.
Pairwise Adjacency Matrix on Spatial Temporal Graph Convolution Network for Skeleton-Based Two-Person Interaction Recognition

Spatial-temporal graph convolutional networks (ST-GCN) have achieved outstanding performances on human action recognition, however, it might be less superior on a two-person interaction recognition (TPIR) task due to the relationship of each skeleton is not considered. In this study, we present an improvement of the STGCN model that focused on TPIR by employing the pairwise adjacency matrix to capture the relationship of person-person skeletons (ST-GCN-PAM). To validate the effectiveness of the proposed ST-GCN-PAM model on TPIR, experiments were conducted on NTU RGB+D120. Additionally, the model was also examined on the Kinetics dataset and NTU RGB+D60. The results show that the proposed ST-GCN-PAM outperforms the-state-of-the-art methods on mutual action of NTU RGB+D120 by achieving 83.28% (cross-subject) and 88.31% (cross-view) accuracy. The model is also superior to the original ST-GCN on the multi-human action of the Kinetics dataset by achieving 41.68% in Top-l and 88.91% in Top-5.
Expose Your Mask: Smart Ponzi Schemes Detection on Blockchain

The anonymity of blockchain has caused Ponzi schemes to be transferred to smart contract platforms by scammers. These Ponzi schemes wearing the mask of smart contracts caused huge losses to people, which makes the detection of smart Ponzi schemes attract people's attention. Recent methods mainly focus on machine learning technology to enable automatic detection for smart Poniz schemes. However, there are some problems with their methods. Firstly, the gradient boosting algorithm in machine learning they used have the problem of prediction shift due to target leakage when processing category features and calculating gradient estimates. Secondly, they ignored the imbalance and repetitiveness of Ponzi schemes on smart contract platforms. These problems can directly lead to model overfitting and affect the generalization ability of trained models. This paper proposes a novel Ponzi schemes detection method on smart contract platform for blockchain. Our method addresses the above issues with the following strategies. Firstly, we leverage ordered target statistic (TS) to process the category features of smart contract. Secondly, we solve the imbalance of dataset through a data augmentation method. Thirdly, with the idea of ordered boosting algorithm, we train a PonziTect model to fight prediction shift caused by target leakage. Based on the above ideas, the experimental results fully manifest the effectiveness and reliability of our model in detecting smart Ponzi schemes on the blockchain. Specifically, our model achieves 98% F-score on the real-world dataset, which significantly outperforms the existing methods. Using our method, we estimate that there are about 532 Ponzi schemes on Ethereum.
Efficient Pig Counting in Crowds with Keypoints Tracking and Spatial-aware Temporal Response Filtering

Pig counting is a crucial task for large-scale pig farming. Pigs are usually visually counted by human. But this process is very time-consuming and error-prone. Few studies in literature developed automated pig counting method. The existing works only focused on pig counting using single image, and its level of accuracy faced challenges due to pig movements, occlusion and overlapping. Especially, the field of view of a single image is very limited, and could not meet the needs of pig counting for large pig grouping houses. Towards addressing these challenges, we presented a real-time automated pig counting system in crowds using only one monocular fisheye camera with an inspection robot. Our system showed that it achieved performance superior to human. Our pipeline began with a novel bottom-up pig detection algorithm to avoid false negatives due to overlapping, occlusion and deformable pig shapes. This detection included a deep convolution neural network (CNN) for pig body part keypoints detection and the keypoints association method to identify individual pigs. It then employed an efficient on-line tracking method to associate pigs across image frames. Finally, pig counts were estimated by a novel spatial-aware temporal response filtering (STRF) method to suppress false positives caused by pig or camera movements or tracking failures. The whole pipeline has been deployed in an edge computing device, and demonstrated the effectiveness.
Artificial Intelligence-Based Multiclass Classification of Benign or Malignant Mucosal Lesions of the Stomach

Gastric cancer (GC) is one of the leading causes of cancer-related death worldwide. It takes some time from chronic gastritis to develop in GC. Early detection of GC will help patients obtain timely treatment. Understanding disease evolution is crucial for the prevention and treatment of GC. Here, we present a convolutional neural network (CNN)-based system to detect abnormalities in the gastric mucosa. We identified normal mucosa, chronic gastritis, and intestinal-type GC: this is the most common route of gastric carcinogenesis. We integrated digitalizing histopathology of whole-slide images (WSIs), stain normalization, a deep CNN, and a random forest classifier. The staining variability of WSIs was reduced significantly through stain normalization, and saved the cost and time of preparing new slides. Stain normalization improved the effect of the CNN model. The accuracy rate at the patch-level reached 98.4%, and 94.5% for discriminating normal → chronic gastritis → GC. The accuracy rate at the WSIs-level for discriminating normal tissue and cancerous tissue reached 96.0%, which is a state-of-the-art result. Survival analyses indicated that the features extracted from the CNN exerted a significant impact on predicting the survival of cancer patients. Our CNN model disclosed significant potential for adjuvant diagnosis of gastric diseases, especially GC, and usefulness for predicting the prognosis.
Data-driven proxy model for waterflood performance prediction and optimization using Echo State Network with Teacher Forcing in mature fields

Waterflood has been widely applied across the world as the most important secondary recovery process to improve reservoir performance. The schemes applied to waterflood projects involve key decision-making which aims to maximize the net present value (NPV) for a given period, improve recovery and sweep efficiency, maintain reservoir pressure, reduce water production, and avoid water recycling. Traditionally, the preferred approach to estimate the future performance of any given waterflood project is through the numerical reservoir simulation. However, despite the great power and intuition associated with the simulation models, sometimes this potential is unachievable due to massive computational requirements. Also, in some other cases, there may exist no simulation model to begin with. To tackle these problems, data-driven proxy models are proposed. In this research, we propose a novel machine-learning-based proxy model for waterflood performance prediction and further apply it for production optimization purpose to obtain the optimal future well control. The data-driven model is realized using the Echo State Network (ESN) under the paradigm of “Reservoir Computing” (RC). Compared with traditional ESNs, this specific methodology includes a feedback loop from the output into the high dimensional “reservoir” to achieve improved prediction results for separated phase production rates. Teacher Forcing (TF) technique is used for easier incorporation of the feedback information without introducing additional recurrent loops during the training process. Furthermore, training ESN proxy models utilizes Ridge Regression, thus all calculations within have analytical solutions that guarantee much improved speed when evaluating forward model-runs, which further lowers the computational demand during the optimization process. The proposed workflow is typically more suitable for mature fields since reliable production data after breakthrough from each producer could greatly improve the training process, and it can be used under the circumstance where no reservoir model has been established. In this research, we present two test examples where we apply ESN to learn the reservoir simulation results of waterflood and further perform open-loop optimization on each of them.
CNN-based Multiple Manipulation Detector Using Frequency Domain Features of Image Residuals

Increasingly sophisticated image editing tools make it easy to modify images. Often these modifications are elaborate, convincing, and undetectable by even careful human inspection. These considerations have prompted the development of forensic algorithms and approaches to detect modifications done to an image. However, these detectors are model-driven (i.e., manipulation-specific) and the choice of a potent detector requires knowledge of the type of manipulation, something that cannot be known ( a priori ). Thus, the latest effort is directed towards developing model-free (i.e., generalized) detectors capable of detecting multiple manipulation types. In this article, we propose a novel detector capable of exposing seven different manipulation types in low-resolution compressed images. Our proposed approach is based on a two-layer convolutional neural network (CNN) to extract frequency domain features of image median filtered residual that are classified using two different classifiers—softmax and extremely randomized trees. Extensive experiments demonstrate the efficacy of proposed detector over existing state-of-the-art detectors.
Hybrid Two-Stage Architecture for Tampering Detection of Chipless ID Cards

Identity verification systems are widely used in daily life. Most of these systems rely on official documents containing identifying information about a person (i.e., passports, ID cards, driving licenses, amongst others). In this kind of approach, the identifiable data is contained inside the embedded chip in the ID card, and can be read remotely by an NFC-enabled mobile device and then matched with a frontal face photograph (selfie) of the person in question. Unfortunately, this method is limited in South-American countries, since only a few of them provide national ID cards that include embedded chips with the owner's identifiable information. For instance, in countries such as Brazil-with a population of over 210 million people-the National ID card does not contain an embedded chip. This work explores a two-stage method, using deep learning techniques, to determine whether an ID card image provided remotely by the user is real, or tampered in the digital (composite) or non-digital domain (high-quality printed or digitally displayed on a screen). Furthermore, RGB images, frequency domain representation, noise features, and error level analysis are tested as different inputs to the two-stage classifier. The proposed BasicNet with Discrete Fourier Transform achieves the highest classification rates of 0.975 for real ID card images, and a mean of 0.968 for fake ID card images.
Reasoning about When to Provide Explanation for Human-involved Self-Adaptive Systems

Many self-adaptive systems benefit from human involvement, where a human operator can provide expertise not available to the system and perform adaptations involving physical changes that cannot be automated. However, a lack of transparency and intelligibility of system goals and the autonomous behaviors enacted to achieve them may hinder a human operator's effort to make such involvement effective. Explanation is sometimes helpful to allow the human to understand why the system is making certain decisions. However, explanations come with costs in terms of, e.g., delayed actions. Hence, it is not always obvious whether explanations will improve the satisfaction of system goals and, if so, when to provide them to the operator. In this work, we define a formal framework for reasoning about explanations of adaptive system behaviors and the conditions under which they are warranted. Specifically, we characterize explanations in terms of their impact on a human operator's ability to effectively engage in adaptive actions. We then present a decision-making approach for planning in self-adaptation that leverages a probabilistic reasoning tool to determine when the explanation should be used in an adaptation strategy in order to improve overall system utility. We illustrate our approach in a representative scenario for the application of an adaptive news website in the context of potential denial-of-service attacks.
One-Shot Learning for Surveillance Anomaly Recognition using Siamese 3D CNN

One-shot image recognition has been explored for many applications in computer vision community. However, its applications in video analytics is not deeply investigated yet. For instance, surveillance anomaly recognition is an open challenging problem and one of its hurdles is the lack of accurate temporally annotated data. This paper addresses the lack of data issue using one-shot learning strategy and proposes an anomaly recognition framework which exploits a 3D CNN siamese network that yields the similarity between two anomaly sequences. This paper also investigates the existing 3D CNNs for this task and then proposes a lightweight 3D CNN model that efficiently handles one-shot anomaly recognition. Once our network is trained, then we can use the powerful discriminative 3D CNN features to predict anomalies not only for the new data but also for entirely new classes. The proposed model is trained using temporally annotated test set of UCF Crime dataset. Finally, the trained model is used to recognize the anomalies and produce temporal automatic labels for the video level weakly annotated training set of the dataset.
A Temporal Pre-Filter For Video Coding Based On Bilateral Filtering

This paper presents a motion compensated temporal bilateral denoising pre-filter for video coding. The filtering process is applied before encoding and uses the two closest preceding pictures, the two closest succeeding pictures, the position of the current picture in the Group of Pictures (GOP) hierarchy and the value of the Quantization Parameter (QP) to filter a current picture. The filter is designed for both random-access and low-delay configurations, and succeeding pictures are not used in low-delay configurations. The filter is reported to achieve an average luma BD-rate of -3.9% relative to the VTM-7.0 video encoder in random-access configuration. The subjective quality compared to VTM-7.0 is reported as similar or better at lower encoded bit rate.