Reviewerly/exHarmony huggingface dataset:
Exploiting Better Feature Aggregation for Video Object Detection

Video object detection (VOD) has been a rising topic in recent years due to the challenges such as occlusion, motion blur, etc. To deal with these challenges, feature aggregation from local or global support frames is verified effective. To exploit better feature aggregation, in this paper, we propose two improvements over previous works: a class-constrained spatial-temporal relation network and a correlation-based feature alignment module. For the class constrained spatial-temporal relation network, it operates on object region proposals, and learns two kinds of relations: (1) the dependencies among region proposals of the same object class from support frames sampled in a long time range or even the whole sequence, and (2) spatial relations among proposals of different objects in the target frame. The homogeneity constraint in spatial-temporal relation network not only filters out many defective proposals but also implicitly embeds the traditional post-processing strategies (e.g., Seq-NMS), leading to a unified end-to-end training networks. In the feature alignment module, we propose a correlation based feature alignment method to align the support and target frames for feature aggregation in the temporal domain. Our experiments show that the proposed method improves the accuracy of single-frame detectors significantly, and outperforms previous temporal or spatial relation networks. Without bells or whistles, the proposed method achieves state-of-the-art performance on the ImageNet VID dataset (84.80% with ResNet-101) without any post-processing methods.
The modeling of human facial pain intensity based on Temporal Convolutional Networks trained with video frames in HSV color space

An accurate detection and management of pain, measured through its relative intensity, plays an important role in the treatment of disease and reducing a patient’s discomfort. As it is relatively difficult to assess, describe, evaluate and manage the pain level using a patient’s self-report, automated pain-detecting tools can provide useful information to assist in the management of pain intensity. This study proposes a new predictive modeling framework that employs a modified Temporal Convolutional Network (TCN) algorithm to recognize the pain intensity prevalent in patients’ video frames collected as part of UNBC-McMaster Shoulder Pain Archive and MIntPAIN databases. The inputs of the proposed TCN network is composed of the extracted and reduced face image features from a fine-tuned VGG-Face and principal component analysis (PCA) with Hue, Saturation, Value (HSV) color spaces video images. The results of TCN based predictive model, employing a long short-term memory (LSTM) model as well as other state-of-the art models, show that the proposed approach performs faster with a high level of efficiency. This is demonstrated by the low magnitude of error metrics (i.e., Mean Squared Error = 0.0629, Mean Absolute Error = 0.1021, correctness validation results represented by Area under Curve = 85% and accuracy metric = 92.44%). Considering the efficiency of the proposed TCN framework, integrating fine-tuned VGG-Face and PCA with Hue, Saturation, Value (HSV) color spaces video images for pain intensity estimation, the present study affirms that the new method can be adopted as an automatic health informatics tool, mainly for pain detection, and subsequently, implemented in the pain management area.
Adversarial Metric Attack and Defense for Person Re-Identification

Person re-identification (re-ID) has attracted much attention recently due to its great importance in video surveillance. In general, distance metrics used to identify two person images are expected to be robust under various appearance changes. However, our work observes the extreme vulnerability of existing distance metrics to adversarial examples, generated by simply adding human-imperceptible perturbations to person images. Hence, the security danger is dramatically increased when deploying commercial re-ID systems in video surveillance. Although adversarial examples have been extensively applied for classification analysis, it is rarely studied in metric analysis like person re-identification. The most likely reason is the natural gap between the training and testing of re-ID networks, that is, the predictions of a re-ID network cannot be directly used during testing without an effective metric. In this work, we bridge the gap by proposing Adversarial Metric Attack, a parallel methodology to adversarial classification attacks. Comprehensive experiments clearly reveal the adversarial effects in re-ID systems. Meanwhile, we also present an early attempt of training a metric-preserving network, thereby defending the metric against adversarial attacks. At last, by benchmarking various adversarial settings, we expect that our work can facilitate the development of adversarial attack and defense in metric-based applications.
General Framework for Randomized Benchmarking

Randomized benchmarking refers to a collection of protocols that in the past decade have become central methods for characterizing quantum gates. These protocols aim at efficiently estimating the quality of a set of quantum gates in a way that is resistant to state preparation and measurement errors. Over the years many versions have been developed, however a comprehensive theoretical treatment of randomized benchmarking has been missing. In this work, we develop a rigorous framework of randomized benchmarking general enough to encompass virtually all known protocols as well as novel, more flexible extensions. Overcoming previous limitations on error models and gate sets, this framework allows us, for the first time, to formulate realistic conditions under which we can rigorously guarantee that the output of any randomized benchmarking experiment is well described by a linear combination of matrix exponential decays. We complement this with a detailed analysis of the fitting problem associated with randomized benchmarking data. We introduce modern signal processing techniques to randomized benchmarking, prove analytical sample complexity bounds, and numerically evaluate performance and limitations. In order to reduce the resource demands of this fitting problem, we introduce novel, scalable postprocessing techniques to isolate exponential decays, significantly improving the practical feasibility of a large set of randomized benchmarking protocols. These postprocessing techniques overcome shortcomings in efficiency of several previously proposed methods such as character benchmarking and linear-cross entropy benchmarking. Finally, we discuss, in full generality, how and when randomized benchmarking decay rates can be used to infer quality measures like the average fidelity. On the technical side, our work substantially extends the recently developed Fourier-theoretic perspective on randomized benchmarking by making use of the perturbation theory of invariant subspaces, as well as ideas from signal processing.2 MoreReceived 8 December 2020Revised 23 December 2021Accepted 4 March 2022DOI:https://doi.org/10.1103/PRXQuantum.3.020357Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasQuantum benchmarkingQuantum computationQuantum gatesQuantum Information
Attentive Meta-graph Embedding for item Recommendation in heterogeneous information networks

Heterogeneous information network (HIN) has become increasingly popular to be exploited in recommender systems, since it contains abundant semantic information to help generate better recommendations. Most conventional work employs meta-paths to model the rich semantics in the HIN. However, the meta-path as a linear structure is insufficient to express the connections. Recently, several work adopts a graph structure, i.e. meta-graph, to express the complex semantics. However, they treat the contributions of nodes in the meta-graph equally, and no explicit representations for users, items or meta-graph based context are learned in the process. To tackle the above problems, this paper proposes an Attentive Meta-graph Embedding approach for item Recommendation, called AMERec, in HINs. Firstly, we prioritize those highly similar pairwise features in the selection of meta-graph instances. Secondly, we differentiate each node in the meta-graph and learn an embedding for each meta-graph. Thirdly, we consider the differences between user and item pairs based on their meta-graph context, and learn a weight for each meta-graph by leveraging the attention mechanism. Finally, we predict the rating by capturing the low- and high-dimensional interaction information between users, items and their meta-graph based context. Comprehensive experiments on three different datasets show that the proposed method is superior to other comparative methods.
Push the Limit of Acoustic Gesture Recognition

With the flourish of the smart devices and their applications, controlling devices using gestures has attracted increasing attention for ubiquitous sensing and interaction. Recent works use acoustic signals to track hand movement and recognize gestures. However, they suffer from low robustness due to frequency selective fading, interference and insufficient training data. In this work, we propose RobuCIR, a robust contact-free gesture recognition system that can work under different practical impact factors with high accuracy and robustness. RobuCIR adopts frequency-hopping mechanism to mitigate frequency selective fading and avoid signal interference. To further increase system robustness, we investigate a series of data augmentation techniques based on a small volume of collected data to emulate different practical impact factors. The augmented data is used to effectively train neural network models and cope with various influential factors (e.g., gesture speed, distance to transceiver, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">etc</i> .). Our experiment results show that RobuCIR can recognize 15 gestures and outperform state-of-the-art works in terms of accuracy and robustness.
Smart Contract Designs on Blockchain Applications

The rapid increase of the world’s urbanization process has been improving citizens’ quality of living. Combining new technologies of smart government, smart healthcare, smart transportation and other services under a framework of smart city minimizes urbanization challenges. However, these services demand a large data technology to support the infrastructure of smart cities. It is a major benefit to using blockchain technology as a framework to integrate multiple technologies of smart city such as Internet of Thing, big data platforms and smart transportation to enhance the automation, security and decentralization of smart city services. However, querying the blockchain to retrieve a transaction record is one of the major limitations of blockchain systems. The operation requires scanning blockchain ledger searching for results. In this paper, we utilized different smart contract designs to support indexing and querying the blockchain for ride sharing data. Our experiments evaluate the complexity of two smart contract designs, Catalog and Sparse smart contracts for indexing and retrieving data from the blockchain
Ostraka: Secure Blockchain Scaling by Node Sharding

Cryptocurrencies, implemented with blockchain protocols, promise to become a global payment system if they can overcome performance limitations. Rapidly advancing architectures improve on latency and throughput, but most require all participating servers to process all transactions. Several recent works propose to shard the system, such that each machine would only process a subset of the transactions. However, we identify a denial-of-service attack that is exposed by these solutions - an attacker can generate transactions that would overload a single shard, thus delaying processing in the entire system. Moreover, we show that in common scenarios, these protocols require most node operators to process almost all blockchain transactions. We present Ostraka, a blockchain node architecture that shards (parallelizes) the nodes themselves. We prove that replacing a unified node with an Ostraka node does not affect the security of the underlying consensus mechanism. We evaluate analytically and experimentally block propagation and processing in various settings. Ostraka allows nodes in the network to scale, without costly coordination. In our experiments, Ostraka nodes transaction processing rate grows linearly with the addition of resources.
Variational quantum algorithm for estimating the quantum Fisher information

The quantum Fisher information (QFI) quantifies the ultimate precision of estimating a parameter from a quantum state and can be regarded as a reliability measure of a quantum system as a quantum sensor. However, estimation of the QFI for a mixed state is in general a computationally demanding task. In this paper we present a variational quantum algorithm called variational quantum Fisher information estimation (VQFIE) to address this task. By estimating lower and upper bounds on the QFI, based on bounding the fidelity, VQFIE outputs a range in which the actual QFI lies. This result can then be used to variationally prepare the state that maximizes the QFI, for the application of quantum sensing. In contrast to previous approaches, VQFIE does not require knowledge of the explicit form of the sensor dynamics. We simulate the algorithm for a magnetometry setup and demonstrate the tightening of our bounds as the state purity increases. For this example, we compare our bounds with literature bounds and show that our bounds are tighter.
Graph Neural Network for Tag Ranking in Tag-enhanced Video Recommendation

In tag-enhanced video recommendation systems, videos are attached with some tags that highlight the contents of videos from different aspects. Tag ranking in such recommendation systems provides personalized tag lists for videos from their tag candidates. A better tag ranking model could attract users to click more tags, enter their corresponding tag channels, and watch more tag-specific videos, which improves both tag click rate and video watching time. However, most conventional tag ranking models merely concentrate on tag-video relevance or tag-related behaviors, ignoring the rich information in video-related behaviors. We should consider user preferences on both tags and videos. In this paper, we propose a novel Graph neural network based tag ranking (GraphTR) framework on a huge heterogeneous network with video, tag, user and media. We design a novel graph neural network that combines multi-field transformer, GraphSAGE and neural FM layers in node aggregation. We also propose a neighbor-similarity based loss to encode various user preferences into heterogeneous node representations. In experiments, we conduct both offline and online evaluations on a real-world video recommendation system in WeChat Top Stories. The significant improvements in both video and tag related metrics confirm the effectiveness and robustness in real-world tag-enhanced video recommendation. Currently, GraphTR has been deployed on WeChat Top Stories for more than six months. The source codes are in https://github.com/lqfarmer/GraphTR.
Turning Up the Dial

Trust and reputation play a core role in underground cybercrime markets, where participants are anonymous and there is little legal recourse for dispute arbitration. These underground markets exist in tension between two opposing forces: the drive to hide incriminating information, and the trust and stability benefits that greater openness yields. Revealing information about transactions to mitigate scams also provides valuable data about the market. We analyse the first dataset, of which we are aware, about the transactions created and completed on a well-known and high-traffic underground marketplace, Hack Forums, along with the associated threads and posts made by its users over two recent years, from June 2018 to June 2020. We use statistical modelling approaches to analyse the economic and social characteristics of the market over three eras, especially its performance as an infrastructure for trust. In the Set-up era, we observe the growth of users making only one transaction, as well as 'power-users' who make many transactions. In the Stable era, we observe a wide range of activities (including large-scale transfers of intermediate currencies such as Amazon Giftcards) which declines slowly from an initial peak. Finally, we analyse the effects of the Covid-19 pandemic, concluding that while we see a significant increase in transactins across all categories, this reflects a stimulus of the market, rather than a transformation. New users overcome the 'cold start' problem by engaging in low-level currency exchanges to prove their trustworthiness. We observe currency exchange accounts for most contracts, and Bitcoin and PayPal are the preferred payment methods by trading values and number of contracts involved. The market is becoming more centralised over time around influential users and threads, with significant changes observed during the Set-up and Covid-19 eras.
Promote-pl

Process models for software product-line engineering focus on proactive adoption scenarios---that is, building product-line platforms from scratch. They comprise the two phases domain engineering (building a product-line platform) and application engineering (building individual variants), each of which defines various development activities. Established more than two decades ago, these process models are still the de-facto standard for steering the engineering of platforms and variants. However, observations from industrial and open-source practice indicate that the separation between domain and application engineering, with their respective activities, does not fully reflect reality. For instance, organizations rarely build platforms from scratch, but start with developing individual variants that are re-engineered into a platform when the need arises. Organizations also appear to evolve platforms by evolving individual variants, and they use contemporary development activities aligned with technical advances. Recognizing this discrepancy, we present an updated process model for engineering software product lines. We employ a method for constructing process theories, building on recent literature as well as our experiences with industrial partners to identify development activities and the orders in which these are performed. Based on these activities, we synthesize and discuss the new process model, called promote-pl. Also, we explain its relation to modern software-engineering practices, such as continuous integration, model-driven engineering, or simulation testing. We hope that our work offers contemporary guidance for product-line engineers developing and evolving platforms, and inspires researchers to build novel methods and tools aligned with current practice.
ACE: Asynchronous and Concurrent Execution of Complex Smart Contracts

Smart contracts are programmable, decentralized and transparent financial applications. Because smart contract platforms typically support Turing-complete programming languages, such systems are often said to enable arbitrary applications. However, the current permissionless smart contract systems impose heavy restrictions on the types of computations that can be implemented. For example, the globally-replicated and sequential execution model of Ethereum requires low gas limits that make many computations infeasible.
A Systematic Literature Review of the Successors of “NeuroEvolution of Augmenting Topologies”

NeuroEvolution (NE) refers to a family of methods for optimizing Artificial Neural Networks (ANNs) using Evolutionary Computation (EC) algorithms. NeuroEvolution of Augmenting Topologies (NEAT) is considered one of the most influential algorithms in the field. Eighteen years after its invention, a plethora of methods have been proposed that extend NEAT in different aspects. In this article, we present a systematic literature review (SLR) to list and categorize the methods succeeding NEAT. Our review protocol identified 232 papers by merging the findings of two major electronic databases. Applying criteria that determine the paper's relevance and assess its quality, resulted in 61 methods that are presented in this article. Our review article proposes a new categorization scheme of NEAT's successors into three clusters. NEAT-based methods are categorized based on 1) whether they consider issues specific to the search space or the fitness landscape, 2) whether they combine principles from NE and another domain, or 3) the particular properties of the evolved ANNs. The clustering supports researchers 1) understanding the current state of the art that will enable them, 2) exploring new research directions or 3) benchmarking their proposed method to the state of the art, if they are interested in comparing, and 4) positioning themselves in the domain or 5) selecting a method that is most appropriate for their problem.
A guided FP-Growth algorithm for mining multitude-targeted item-sets and class association rules in imbalanced data

Identifying frequent item-sets is a popular data-mining task. It consists of finding sets of items frequently appearing in data. Yet, finding all frequent item-sets in large or dense datasets may be time-consuming, and a user may be interested merely in some specific item-sets rather than all of them. Recently, methods have been proposed for targeted item-set mining; that is to calculate the support of some item-sets of interest. Though this approach is often more suitable for real applications than traditional item-set mining approaches, performance remains an issue. To address that issue, this paper presents a novel algorithm for multitude-targeted mining, named Guided Frequent Pattern-Growth (GFP-Growth). The GFP-Growth algorithm is designed to quickly mine a given set of item-sets using a small amount of memory. This paper proves that GFP-Growth yields the exact frequency-counts for each item-set of interest. It further shows that GFP-Growth can boost the performance for several problems requiring item-set mining. We specifically study the problem of generating minority-class rules from imbalanced data and develop the Minority-Report Algorithm (MRA) that uses GFP-Growth to solve this problem efficiently. We prove several theoretical properties of MRA and present experimental results showing substantial performance gain.
Ontology-Enhanced Machine Learning: A Bosch Use Case of Welding Quality Monitoring

In the automotive industry, welding is a critical process of automated manufacturing and its quality monitoring is important. IoT technologies behind automated factories enable adoption of Machine Learning (ML) approaches for quality monitoring. Development of such ML models requires collaborative work of experts from different areas, including data scientists, engineers, process experts, and managers. The asymmetry of their backgrounds, the high variety and diversity of data relevant for quality monitoring pose significant challenges for ML modeling. In this work, we address these challenges by empowering ML-based quality monitoring methods with semantic technologies. We propose a system, called SemML, for ontology-enhanced ML pipeline development. It has several novel components and relies on ontologies and ontology templates for task negotiation and for data and ML feature annotation. We evaluated SemML on the Bosch use-case of electric resistance welding with very promising results.