Reviewerly/exHarmony huggingface dataset:
Towards a Better Match in Siamese Network Based Visual Object Tracker

Recently, Siamese network based trackers have received tremendous interest for their fast tracking speed and high performance. Despite the great success, this tracking framework still suffers from several limitations. First, it cannot properly handle large object rotation. Second, tracking gets easily distracted when the background contains salient objects. In this paper, we propose two simple yet effective mechanisms, namely angle estimation and spatial masking, to address these issues. The objective is to extract more representative features so that a better match can be obtained between the same object from different frames. The resulting tracker, named Siam-BM, not only significantly improves the tracking performance, but more importantly maintains the realtime capability. Evaluations on the VOT2017 dataset show that Siam-BM achieves an EAO of 0.335, which makes it the best-performing realtime tracker to date.
Multiobjective Support Vector Machines: Handling Class Imbalance With Pareto Optimality

Support vector machines (SVMs) seek to optimize three distinct objectives: maximization of margin, minimization of regularization from the positive class, and minimization of regularization from the negative class. The right choice of weightage for each of these objectives is critical to the quality of the classifier learned, especially in case of the class imbalanced data sets. Therefore, costly parameter tuning has to be undertaken to find a set of suitable relative weights. In this brief, we propose to train SVMs, on two-class as well as multiclass data sets, in a multiobjective optimization framework called radial boundary intersection to overcome this shortcoming. The experimental results suggest that the radial boundary intersection-based scheme is indeed effective in finding the best tradeoff among the objectives compared with parameter-tuning schemes.
Spatio-Temporal Interpolated Echo State Network for Meteorological Series Prediction

Spatio-temporal series prediction has attracted increasing attention in the field of meteorology in recent years. The spatial and temporal joint effect makes predictions challenging. Most of the existing spatio-temporal prediction models are computationally complicated. To develop an accurate but easy-to-implement spatio-temporal prediction model, this paper designs a novel spatio-temporal prediction model based on echo state networks. For real-world observed meteorological data with randomness and large changes, we use a cubic spline method to bridge the gaps between the neighboring points, which results in a pleasingly smooth series. The interpolated series is later input into the spatio-temporal echo state networks, in which the spatial coefficients are computed by the elastic-net algorithm. This approach offers automatic selection and continuous shrinkage of the spatial variables. The proposed model provides an intuitive but effective approach to address the interaction of spatial and temporal effects. To demonstrate the practicality of the proposed model, we apply it to predict two real-world datasets: monthly precipitation series and daily air quality index series. Experimental results demonstrate that the proposed model achieves a normalized root-mean-square error of approximately 0.250 on both datasets. Similar results are achieved on the long short-term memory model, but the computation time of our proposed model is considerably shorter. It can be inferred that our proposed neural network model has advantages on predicting meteorological series over other models.
Learning to Transform, Combine, and Reason in Open-Domain Question Answering

Users seek direct answers to complex questions from large open-domain knowledge sources like the Web. Open-domain question answering has become a critical task to be solved for building systems that help address users' complex information needs. Most open-domain question answering systems use a search engine to retrieve a set of candidate documents, select one or a few of them as context, and then apply reading comprehension models to extract answers. Some questions, however, require taking a broader context into account, e.g., by considering low-ranked documents that are not immediately relevant, combining information from multiple documents, and reasoning over multiple facts from these documents to infer the answer. In this paper, we propose a model based on the Transformer architecture that is able to efficiently operate over a larger set of candidate documents by effectively combining the evidence from these documents during multiple steps of reasoning, while it is robust against noise from low-ranked non-relevant documents included in the set. We use our proposed model, called TraCRNet, on two public open-domain question answering datasets, SearchQA and Quasar-T, and achieve results that meet or exceed the state-of-the-art.
An Improved Particle Swarm Optimization Algorithm with Adaptive Inertia Weights

The particle swarm optimization (PSO) algorithm is simple to implement and converges quickly, but it easily falls into a local optimum; on the one hand, it lacks the ability to balance global exploration and local exploitation of the population, and on the other hand, the population lacks diversity. To solve these problems, this paper proposes an improved adaptive inertia weight particle swarm optimization (AIWPSO) algorithm. The AIWPSO algorithm includes two strategies: (1) An inertia weight adjustment method based on the optimal fitness value of individual particles is proposed, so that different particles have different inertia weights. This method increases the diversity of inertia weights and is conducive to balancing the capabilities of global exploration and local exploitation. (2) A mutation threshold is used to determine which particles need to be mutated. This method compensates for the inaccuracy of random mutation, effectively increasing the diversity of the population. To evaluate the performance of the proposed AIWPSO algorithm, benchmark functions are used for testing. The results show that AIWPSO achieves satisfactory results compared with those of other PSO algorithms. This outcome shows that the AIWPSO algorithm is conducive to balancing the abilities of the global exploration and local exploitation of the population, while increasing the diversity of the population, thereby significantly improving the optimization ability of the PSO algorithm.
Multilayer probability extreme learning machine for device-free localization

Device-free localization (DFL) is becoming one of the new techniques in wireless localization field, due to its advantage that the target to be localized does not need to attach any electronic device. One of the key issues of DFL is how to characterize the influence of the target on the wireless links, such that the target’s location can be accurately estimated by analyzing the changes of the signals of the links. Most of the existing related research works usually extract the useful information from the links through manual approaches, which are labor-intensive and time-consuming. Deep learning approaches have attempted to automatically extract the useful information from the links, but the training of the conventional deep learning approaches are time-consuming, because a large number of parameters need to be fine-tuned multiple times. Motivated by the fast learning speed and excellent generalization performance of extreme learning machine (ELM), which is an emerging training approach for generalized single hidden layer feedforward neural networks (SLFNs), this paper proposes a novel hierarchical ELM based on deep learning theory, named multilayer probability ELM (MP-ELM), for automatically extracting the useful information from the links, and implementing fast and accurate DFL. The proposed MP-ELM is stacked by ELM autoencoders, so it also keeps the very fast learning speed of ELM. In addition, considering the uncertainty and redundant links existing in DFL, MP-ELM outputs the probabilistic estimation of the target’s location instead of the deterministic output. The validity of the proposed MP-ELM-based DFL is evaluated both in the indoor and the outdoor environments, respectively. Experimental results demonstrate that the proposed MP-ELM can obtain better performance compared with classic ELM, multilayer ELM (ML-ELM), hierarchical ELM (H-ELM), deep belief network (DBN), and deep Boltzmann machine (DBM).
Sliced Score Matching: A Scalable Approach to Density and Score Estimation

Score matching is a popular method for estimating unnormalized statistical models. However, it has been so far limited to simple, shallow models or low-dimensional data, due to the difficulty of computing the Hessian of log-density functions. We show this difficulty can be mitigated by projecting the scores onto random vectors before comparing them. This objective, called sliced score matching, only involves Hessian-vector products, which can be easily implemented using reverse-mode automatic differentiation. Therefore, sliced score matching is amenable to more complex models and higher dimensional data compared to score matching. Theoretically, we prove the consistency and asymptotic normality of sliced score matching estimators. Moreover, we demonstrate that sliced score matching can be used to learn deep score estimators for implicit distributions. In our experiments, we show sliced score matching can learn deep energy-based models effectively, and can produce accurate score estimates for applications such as variational inference with implicit distributions and training Wasserstein Auto-Encoders.
Equations reloaded: high-level dependently-typed functional programming and proving in Coq

Equations is a plugin for the Coq proof assistant which provides a notation for defining programs by dependent pattern-matching and structural or well-founded recursion. It additionally derives useful high-level proof principles for demonstrating properties about them, abstracting away from the implementation details of the function and its compiled form. We present a general design and implementation that provides a robust and expressive function definition package as a definitional extension to the Coq kernel. At the core of the system is a new simplifier for dependent equalities based on an original handling of the no-confusion property of constructors.
Estimation of Rotation Gain Thresholds Considering FOV, Gender, and Distractors

Redirected walking techniques enable users to naturally locomote in virtual environments (VEs) that are larger than the tracked space. Redirected walking imperceptibly transforms the VE around the user with predefined estimated threshold gains. Previously estimated gains were evaluated with a 40° field of view (FOV), and have not been evaluated in the presence of a distractor-a moving object in the VE that may capture the user's attention. We conducted a 2 (FOV: 40°, 110°) × 2 (Gender: female, male) × 2 (Distractor: without, with) user study to estimate and compare thresholds for rotation gains. Significant differences in detection thresholds were found between FOVs, and significant differences were found between female and male gains with a 110° FOV. Males had significantly wider gains using a 110° FOV compared to a 40° FOV, and distractors affected females differently than males. Finally, strong correlations were found between simulator sickness scores and threshold gains.
Liver tumor segmentation in CT volumes using an adversarial densely connected network

Abstract Background Malignant liver tumor is one of the main causes of human death. In order to help physician better diagnose and make personalized treatment schemes, in clinical practice, it is often necessary to segment and visualize the liver tumor from abdominal computed tomography images. Due to the large number of slices in computed tomography sequence, developing an automatic and reliable segmentation method is very favored by physicians. However, because of the noise existed in the scan sequence and the similar pixel intensity of liver tumors with their surrounding tissues, besides, the size, position and shape of tumors also vary from one patient to another, automatic liver tumor segmentation is still a difficult task. Results We perform the proposed algorithm to the Liver Tumor Segmentation Challenge dataset and evaluate the segmentation results. Experimental results reveal that the proposed method achieved an average Dice score of 68.4% for tumor segmentation by using the designed network, and ASD, MSD, VOE and RVD improved from 27.8 to 21, 147 to 124, 0.52 to 0.46 and 0.69 to 0.73, respectively after performing adversarial training strategy, which proved the effectiveness of the proposed method. Conclusions The testing results show that the proposed method achieves improved performance, which corroborated the adversarial training based strategy can achieve more accurate and robustness results on liver tumor segmentation task.
Formal verification of a constant-time preserving C compiler

Timing side-channels are arguably one of the main sources of vulnerabilities in cryptographic implementations. One effective mitigation against timing side-channels is to write programs that do not perform secret-dependent branches and memory accesses. This mitigation, known as "cryptographic constant-time", is adopted by several popular cryptographic libraries. This paper focuses on compilation of cryptographic constant-time programs, and more specifically on the following question: is the code generated by a realistic compiler for a constant-time source program itself provably constant-time? Surprisingly, we answer the question positively for a mildly modified version of the CompCert compiler, a formally verified and moderately optimizing compiler for C. Concretely, we modify the CompCert compiler to eliminate sources of potential leakage. Then, we instrument the operational semantics of CompCert intermediate languages so as to be able to capture cryptographic constant-time. Finally, we prove that the modified CompCert compiler preserves constant-time. Our mechanization maximizes reuse of the CompCert correctness proof, through the use of new proof techniques for proving preservation of constant-time. These techniques achieve complementary trade-offs between generality and tractability of proof effort, and are of independent interest.
A Private Blockchain Implementation Using Multichain Open Source Platform

The impact of digital transformation is becoming visible in every aspect of our lives. Digital transformation strategies mostly rely on disruptive technologies such as Internet of Things, augmented reality, artificial intelligence, and blockchain. Blockchain provides decentralized, immutable, and trustless database distributed across all participants and it was initially proposed as distributed ledger for digital cryptocurrency. However, besides in the finance sector, blockchain technology is considered to be a major innovation enabler in various industries. The paper describes an implementation of a private blockchain using the Multichain open source platform with possible application to agrifood use cases such as food tracking and tracing, product lifecycle management, and counterfeit prevention.
3D-Convolutional Neural Network with Generative Adversarial Network and Autoencoder for Robust Anomaly Detection in Video Surveillance

As the surveillance devices proliferate, various machine learning approaches for video anomaly detection have been attempted. We propose a hybrid deep learning model composed of a video feature extractor trained by generative adversarial network with deficient anomaly data and an anomaly detector boosted by transferring the extractor. Experiments with UCSD pedestrian dataset show that it achieves 94.4% recall and 86.4% precision, which is the competitive performance in video anomaly detection.
Contextualized Emotion Recognition in Conversation as Sequence Tagging

Emotion recognition in conversation (ERC) is an important topic for developing empathetic machines in a variety of areas including social opinion mining, health-care and so on. In this paper, we propose a method to model ERC task as sequence tagging where a Conditional Random Field (CRF) layer is leveraged to learn the emotional consistency in the conversation. We employ LSTM-based encoders that capture self and inter-speaker dependency of interlocutors to generate contextualized utterance representations which are fed into the CRF layer. For capturing long-range global context, we use a multi-layer Transformer encoder to enhance the LSTM-based encoder. Experiments show that our method benefits from modeling the emotional consistency and outperforms the current state-of-the-art methods on multiple emotion classification datasets.
Understanding Multimodal User Gesture and Speech Behavior for Object Manipulation in Augmented Reality Using Elicitation

The primary objective of this research is to understand how users manipulate virtual objects in augmented reality using multimodal interaction (gesture and speech) and unimodal interaction (gesture). Through this understanding, natural-feeling interactions can be designed for this technology. These findings are derived from an elicitation study employing Wizard of Oz design aimed at developing user-defined multimodal interaction sets for building tasks in 3D environments using optical see-through augmented reality headsets. The modalities tested are gesture and speech combined, gesture only, and speech only. The study was conducted with 24 participants. The canonical referents for translation, rotation, and scale were used along with some abstract referents (create, destroy, and select). A consensus set of gestures for interactions is provided. Findings include the types of gestures performed, the timing between co-occurring gestures and speech (130 milliseconds), perceived workload by modality (using NASA TLX), and design guidelines arising from this study. Multimodal interaction, in particular gesture and speech interactions for augmented reality headsets, are essential as this technology becomes the future of interactive computing. It is possible that in the near future, augmented reality glasses will become pervasive.
Detection of Disaster-Affected Cultural Heritage Sites from Social Media Images Using Deep Learning Techniques

This article describes a method for early detection of disaster-related damage to cultural heritage. It is based on data from social media, a timely and large-scale data source that is nevertheless quite noisy. First, we collect images posted on social media that may refer to a cultural heritage site. Then, we automatically categorize these images according to two dimensions: whether they are indeed a photo in which a cultural heritage resource is the main subject, and whether they represent damage. Both categorizations are challenging image classification tasks, given the ambiguity of these visual categories; we tackle both tasks using a convolutional neural network. We test our methodology on a large collection of thousands of images from the web and social media, which exhibit the diversity and noise that is typical of these sources, and contain buildings and other architectural elements, heritage and not-heritage, damaged by disasters as well as intact. Our results show that while the automatic classification is not perfect, it can greatly reduce the manual effort required to find photos of damaged cultural heritage by accurately detecting relevant candidates to be examined by a cultural heritage professional.
Program Enhanced Fact Verification with Verbalization and Graph Attention Network

Performing fact verification based on structured data is important for many real-life applications and is a challenging research problem, particularly when it involves both symbolic operations and informal inference based on language understanding. In this paper, we present a Program-enhanced Verbalization and Graph Attention Network (ProgVGAT) to integrate programs and execution into textual inference models. Specifically, a verbalization with program execution model is proposed to accumulate evidences that are embedded in operations over the tables. Built on that, we construct the graph attention verification networks, which are designed to fuse different sources of evidences from verbalized program execution, program structures, and the original statements and tables, to make the final verification decision. To support the above framework, we propose a program selection module optimized with a new training strategy based on margin loss, to produce more accurate programs, which is shown to be effective in enhancing the final verification results. Experimental results show that the proposed framework achieves the new state-of-the-art performance, a 74.4% accuracy, on the benchmark dataset TABFACT.
CAT-Gen: Improving Robustness in NLP Models via Controlled Adversarial Text Generation

NLP models are shown to suffer from robustness issues, i.e., a model’s prediction can be easily changed under small perturbations to the input. In this work, we present a Controlled Adversarial Text Generation (CAT-Gen) model that, given an input text, generates adversarial texts through controllable attributes that are known to be invariant to task labels. For example, in order to attack a model for sentiment classification over product reviews, we can use the product categories as the controllable attribute which would not change the sentiment of the reviews. Experiments on real-world NLP datasets demonstrate that our method can generate more diverse and fluent adversarial texts, compared to many existing adversarial text generation approaches. We further use our generated adversarial examples to improve models through adversarial training, and we demonstrate that our generated attacks are more robust against model re-training and different model architectures.
Design and Implementation of Deep Learning Based Contactless Authentication System Using Hand Gestures

Hand gestures based sign language digits have several contactless applications. Applications include communication for impaired people, such as elderly and disabled people, health-care applications, automotive user interfaces, and security and surveillance. This work presents the design and implementation of a complete end-to-end deep learning based edge computing system that can verify a user contactlessly using ‘authentication code’. The ‘authentication code’ is an ‘n’ digit numeric code and the digits are hand gestures of sign language digits. We propose a memory-efficient deep learning model to classify the hand gestures of the sign language digits. The proposed deep learning model is based on the bottleneck module which is inspired by the deep residual networks. The model achieves classification accuracy of 99.1% on the publicly available sign language digits dataset. The model is deployed on a Raspberry pi 4 Model B edge computing system to serve as an edge device for user verification. The edge computing system consists of two steps, it first takes input from the camera attached to it in real-time and stores it in the buffer. In the second step, the model classifies the digit with the inference rate of 280 ms, by taking the first image in the buffer as input.