Reviewerly/exHarmony huggingface dataset:
Kalman Filter for Spatial-Temporal Regularized Correlation Filters

We consider visual tracking in numerous applications of computer vision and seek to achieve optimal tracking accuracy and robustness based on various evaluation criteria for applications in intelligent monitoring during disaster recovery activities. We propose a novel framework to integrate a Kalman filter (KF) with spatial-temporal regularized correlation filters (STRCF) for visual tracking to overcome the instability problem due to large-scale application variation. To solve the problem of target loss caused by sudden acceleration and steering, we present a stride length control method to limit the maximum amplitude of the output state of the framework, which provides a reasonable constraint based on the laws of motion of objects in real-world scenarios. Moreover, we analyze the attributes influencing the performance of the proposed framework in large-scale experiments. The experimental results illustrate that the proposed framework outperforms STRCF on OTB-2013, OTB-2015 and Temple-Color datasets for some specific attributes and achieves optimal visual tracking for computer vision. Compared with STRCF, our framework achieves AUC gains of 2.8%, 2%, 1.8%, 1.3%, and 2.4% for the background clutter, illumination variation, occlusion, out-of-plane rotation, and out-of-view attributes on the OTB-2015 datasets, respectively. For sporting events, our framework presents much better performance and greater robustness than its competitors.
A Generalized Deep Neural Network Approach for Digital Watermarking Analysis

Technology advancement has facilitated digital content, such as images, being acquired in large volumes. However, requirement from the privacy or legislation perspective still demands the need for intellectual content protection. In this paper, we propose a deep neural network (DNN) based watermarking method to achieve this goal. Instead of training a neural network for protecting a specific image, we train the network on an image dataset and generalize the trained model to protect distinct test images in a bulk manner. Respective evaluations from both the subjective and objective aspects confirm the generality and practicality of our proposed method. To demonstrate the robustness of this general neural watermarking approach, commonly used attacks are applied to the watermarked images to examine the corresponding extracted watermarks, which still retain sufficient recognizable traits for some occasions. Testing on distinctive dataset shows the satisfying generalization of our proposed method, and practice such as loss function adjustment can cater to the capacity requirement of complicated watermark. We also discuss some traits of the trained model, which incur the vulnerability to JPEG compression attack. However, remedy seeking for this can potentially open a window to understand the underlying working principle of DNN in future work. Considering its performance and economy, it is concluded that subsequent studies that generalize our work on utilizing DNN for intellectual content protection might be a promising research trend.
Learning Universal Network Representation via Link Prediction by Graph Convolutional Neural Network

Network representation learning algorithms, which aim at automatically encoding graphs into low-dimensional vector representations with a variety of node similarity definitions, have a wide range of downstream applications. Most existing methods either have low accuracies in downstream tasks or a very limited application field, such as article classification in citation networks. In this paper, we propose a novel network representation method, named Link Prediction based Network Representation (LPNR), which generalizes the latest graph neural network and optimizes a carefully designed objective function that preserves linkage structures. LPNR can not only learn meaningful node representations that achieve competitive accuracy in node centrality measurement and community detection but also achieve high accuracy in the link prediction task. Experiments prove the effectiveness of LPNR on three real-world networks. With the mini-batch and fixed sampling strategy, LPNR can learn the embedding of large graphs in a few hours.
GPTune

Multitask learning has proven to be useful in the field of machine learning when additional knowledge is available to help a prediction task. We adapt this paradigm to develop autotuning frameworks, where the objective is to find the optimal performance parameters of an application code that is treated as a black-box function. Furthermore, we combine multitask learning with multi-objective tuning and incorporation of coarse performance models to enhance the tuning capability. The proposed framework is parallelized and applicable to any application, particularly exascale applications with a small number of function evaluations. Compared with other state-of-the-art single-task learning frameworks, the proposed framework attains up to 2.8X better code performance for at least 80% of all tasks using up to 2048 cores.
Image Compositing for Segmentation of Surgical Tools Without Manual Annotations

Producing manual, pixel-accurate, image segmentation labels is tedious and time-consuming. This is often a rate-limiting factor when large amounts of labeled images are required, such as for training deep convolutional networks for instrument-background segmentation in surgical scenes. No large datasets comparable to industry standards in the computer vision community are available for this task. To circumvent this problem, we propose to automate the creation of a realistic training dataset by exploiting techniques stemming from special effects and harnessing them to target training performance rather than visual appeal. Foreground data is captured by placing sample surgical instruments over a chroma key (a.k.a. green screen) in a controlled environment, thereby making extraction of the relevant image segment straightforward. Multiple lighting conditions and viewpoints can be captured and introduced in the simulation by moving the instruments and camera and modulating the light source. Background data is captured by collecting videos that do not contain instruments. In the absence of pre-existing instrument-free background videos, minimal labeling effort is required, just to select frames that do not contain surgical instruments from videos of surgical interventions freely available online. We compare different methods to blend instruments over tissue and propose a novel data augmentation approach that takes advantage of the plurality of options. We show that by training a vanilla U-Net on semi-synthetic data only and applying a simple post-processing, we are able to match the results of the same network trained on a publicly available manually labeled real dataset.
Label Efficient Visual Abstractions for Autonomous Driving

It is well known that semantic segmentation can be used as an effective intermediate representation for learning driving policies. However, the task of street scene semantic segmentation requires expensive annotations. Furthermore, segmentation algorithms are often trained irrespective of the actual driving task, using auxiliary image-space loss functions which are not guaranteed to maximize driving metrics such as safety or distance traveled per intervention. In this work, we seek to quantify the impact of reducing segmentation annotation costs on learned behavior cloning agents. We analyze several segmentation-based intermediate representations. We use these visual abstractions to systematically study the trade-off between annotation efficiency and driving performance, i.e., the types of classes labeled, the number of image samples used to learn the visual abstraction model, and their granularity (e.g., object masks vs. 2D bounding boxes). Our analysis uncovers several practical insights into how segmentation-based visual abstractions can be exploited in a more label efficient manner. Surprisingly, we find that state-of-the-art driving performance can be achieved with orders of magnitude reduction in annotation cost. Beyond label efficiency, we find several additional training benefits when leveraging visual abstractions, such as a significant reduction in the variance of the learned policy when compared to state-of-the-art end-to-end driving models.
Physical Layer Dynamic Key Encryption in OFDM-PON System Based on Cellular Neural Network

In this paper, we propose a dynamic key technique based on Cellular Neural Network (CNN) for security improvement in the orthogonal frequency division multiplexing passive optical network (OFDM-PON). To enhance the encryption scheme security, a six-dimensional CNN hyperchaotic system is employed to encrypt the data. And, the keys are divided into the dynamic and static. The dynamic key is randomly extracted from a key set by incorporating the random feature of the input data. Then, the chaotic sequence generated by the dynamic key is served as the synchronous sequence for encryption. Moreover, the chaotic sequences generated by the static keys are used to resist the chosen-plaintext attacks (CPAs) and scramble the phase of QAM symbols on the frequency domain. With these processing techniques, the multi-fold data encryption can create a key space of ~10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">315</sup> to protect against the exhaustive trial. The transmission of 10-Gb/s encrypted 16-QAM-based OFDM signal is demonstrated over 20-km single mode fiber (SMF) by experiment. The results show that our proposed scheme can provide excellent confidentiality of data transmission against the CPAs and brute-force attack.
Deblurring Dynamic Scenes via Spatially Varying Recurrent Neural Networks

Deblurring images captured in dynamic scenes is challenging as the motion blurs are spatially varying caused by camera shakes and object movements. In this paper, we propose a spatially varying neural network to deblur dynamic scenes. The proposed model is composed of three deep convolutional neural networks (CNNs) and a recurrent neural network (RNN). The RNN is used as a deconvolution operator on feature maps extracted from the input image by one of the CNNs. Another CNN is used to learn the spatially varying weights for the RNN. As a result, the RNN is spatial-aware and can implicitly model the deblurring process with spatially varying kernels. To better exploit properties of the spatially varying RNN, we develop both one-dimensional and two-dimensional RNNs for deblurring. The third component, based on a CNN, reconstructs the final deblurred feature maps into a restored image. In addition, the whole network is end-to-end trainable. Quantitative and qualitative evaluations on benchmark datasets demonstrate that the proposed method performs favorably against the state-of-the-art deblurring algorithms.
Permissioned Blockchain Through the Looking Glass: Architectural and Implementation Lessons Learned

Since the inception of Bitcoin, the distributed systems community has shown interest in the design of efficient blockchain systems. However, initial blockchain applications (like Bitcoin) attain very low throughput, which has promoted the design of permissioned blockchain systems. These permissioned blockchain systems employ classical Byzantine-Fault Tolerant (BFT) protocols to reach consensus. However, existing permissioned blockchain systems still attain low throughputs (of the order 10K txns/s). As a result, existing works blame this low throughput on the associated BFT protocol and expend resources in developing optimized protocols. We believe such blames only depict a one-sided story. In specific, we raise a simple question: can a well-crafted system based on a classical BFT protocol outperform a modern protocol? We show that designing such a well-crafted system is possible and illustrate that even if such a system employs a three-phase protocol, it can outperform another system utilizing a single-phase protocol. This endeavor requires us to dissect a permissioned blockchain system and highlight different factors that affect its performance. Based on our insights, we present the design of our enterprise-grade, high-throughput yielding permissioned blockchain system, ResilientDB, that employs multi-threaded deep pipelines, to balance tasks at a replica, and provides guidelines for future designs.
MaskHunter: real‐time object detection of face masks during the COVID‐19 pandemic

Due to the COVID-19 pandemic at present, it is necessary to detect whether pedestrians in public places wear face masks or not for preventing the spread of novel coronavirus. The pedestrian flow in public places is large, and it puts forward higher requirements for the accuracy and speed of real-time mask detection. Improving the face mask detection effect especially in the night environment is a challenging problem. A novel object detector namely MaskHunter is proposed in this study for the real-time mask detection. Specifically, the authors propose novel effective structures of backbone, neck and prediction head based on YOLOv4 series, which achieves the state-of-the-art performance and a novel improved Mosaic data augmentation method. Moreover, they propose a novel mask-guided module to enhance the discrimination ability of face mask especially in the night environment. As a consequent, experiments show that MaskHunter achieves better detection performance for real-time mask detection compared with other obtained models in this scenario.
Improving knowledge distillation via an expressive teacher

Knowledge distillation (KD) is a widely used network compression technique for seeking a light student network with similar behaviors to its heavy teacher network. Previous studies mainly focus on training the student to mimic representation space of the teacher. However, how to be a good teacher is rarely explored. We find that if a teacher has weak ability to capture the knowledge underlying the true data in the real world, the student cannot even learn knowledge from its teacher. Inspired by that, we propose an inter-class correlation regularization to train teacher to capture a more explicit correlation among classes. Besides, we enforce student to mimic inter-class correlation of its teacher. Extensive experiments of image classification task have been conducted on four public benchmarks. For example, when the teacher and student networks are ShuffleNetV2-1.0 and ShuffleNetV2-0.5, our proposed method achieves 42.63% top-1 error rate for Tiny ImageNet.
Time-and-Concept Enhanced Deep Multidimensional Item Response Theory for interpretable Knowledge Tracing

Knowledge Tracing (KT), namely tracking the knowledge conditions of each student across time, has always been challenging due to the latent and time-varying characteristics of knowledge states. Traditional psychometrical frameworks lack the ability to extract rich representations of exercises or examinees. Deep learning KT models have shown comparable performances, but uninterpretable model parameters have limited their applications. Furthermore, existing frameworks usually cannot handle temporal factors appropriately, as most of them simply apply stochastic processes to simulate fluctuations on knowledge states over time. In this paper, we propose a new framework named Time-and-Concept Enhanced Deep Multidimensional Item Response Theory (TC-MIRT) that integrates the parameters of a Multidimensional Item Response Theory into an improved recurrent neural network. Specifically, two enhanced components are constructed to empower our model with the ability to perform trend forecasting and to generate explainable parameters in each specific domain of knowledge. Experiments implemented on two real-world datasets show that our framework outperforms state-of-the-art KT approaches on performance prediction tasks. Moreover, extensive case analyses also demonstrate that the interpretable parameters of TC-MIRT can be used to evaluate the strengths and weaknesses of students.
YOLO Based Real-Time Human Detection for Smart Video Surveillance at the Edge

Recently, smart video surveillance at the edge has become a trend in developing security applications since edge computing enables more image processing tasks to be implemented on the decentralised network note of the surveillance system. As a result, many security applications such as behaviour recognition and prediction, employee safety, perimeter intrusion detection and vandalism deterrence can minimise their latency or even process in real-time when the camera network system is extended to a larger degree. Technically, human detection is a key step in the implementation of these applications. With the advantage of high detection rates, deep learning methods have been widely employed on edge devices in order to detect human objects. However, due to their high computation costs, it is challenging to apply these methods on resource limited edge devices for real-time applications. Inspired by the You Only Look Once (YOLO), residual learning and Spatial Pyramid Pooling (SPP), a novel form of real-time human detection is presented in this paper. Our approach focuses on designing a network structure so that the developed model can achieve a good trade-off between accuracy and processing time. Experimental results show that our trained model can process 2 FPS on Raspberry PI 3B and detect humans with accuracies of 95.05 % and 96.81 % when tested respectively on INRIA and PENN FUDAN datasets. On the human COCO test dataset, our trained model outperforms the performance of the Tiny-YOLO versions. Additionally, compare to the SSD based L-CNN method, our algorithm achieves better accuracy than the other method.
GEMS: GPU-Enabled Memory-Aware Model-Parallelism System for Distributed DNN Training

Data-parallelism has become an established paradigm to train DNNs that fit inside GPU memory on large-scale HPC systems. However, model-parallelism is required to train out-of-core DNNs. In this paper, we deal with emerging requirements brought forward by very large DNNs being trained using high-resolution images common in digital pathology. To address these, we propose, design, and implement GEMS; a GPU-Enabled Memory-Aware Model-Parallelism System. We present several design schemes like GEMS-MAST, GEMS-MASTER, and GEMS-Hybrid that offer excellent speedups over state-of-the-art systems like Mesh-TensorFlow and FlexFlow. Furthermore, we combine model-parallelism and data-parallelism to train a 1000-1ayer ResNet-lk model using 1,024 Volta V100 GPUs with 97.32% scaling-efficiency. For the real-world histopathology whole-slide-image (WSI) of 100,000 x 100,000 pixels, we train custom ResNet-110-v2 on image tiles of size 1024 x 1024 and reduce the training time from seven hours to 28 minutes.
Adaptive cross-contextual word embedding for word polysemy with unsupervised topic modeling

Because of its efficiency, word embedding has been widely used in many natural language processing and text modeling tasks. It aims to represent each word by a vector so such that the geometry between these vectors can capture the semantic correlations between words. An ambiguous word can often have diverse meanings in different contexts, a quality which is called polysemy. The bulk of studies aimed to generate only one single embedding for each word, whereas a few studies have made a small number of embeddings to present different meanings of each word. However, it is hard to determine the exact number of senses for each word, as meanings depend on contexts. To address this problem, this paper proposes a novel adaptive cross-contextual word embedding (ACWE) method for capturing the word polysemy in different contexts based on topic modeling, in which the word polysemy is defined over a latent interpretable semantic space. The proposed ACWE consists of two main parts, in the first of which an unsupervised cross-contextual probabilistic word embedding model is designed to obtain the global word embeddings, and each word is represented by an embedding in the unified latent semantic space. Based on the global word embeddings, an adaptive cross-contextual word embedding process is then devised in the second part to learn the local embeddings for each polysemous word in different contexts. In fact, a word embedding is adaptively adjusted and updated with respect to different contexts to generate different word embeddings tailored to the corresponding contexts. The proposed ACWE is validated on two datasets collected from Wikipedia and IMDb on different tasks including word similarity, polysemy induction, semantic interpretability, and text classification. Experimental results indicate that ACWE does not only outperform the established word embedding methods, which consider word polysemy on six popular benchmark datasets, but it also yields competitive performance compared with state-of-the-art deep learning-based approaches without considering polysemy. Moreover, the proposed ACWE significantly improves the performances of text classification both in precision and F1, and the visualizations of the semantics of words demonstrate the feasibility and advantage of the proposed ACWE model on polysemy.
Blockchain and Edge Computing for Decentralized EMRs Sharing in Federated Healthcare

Blockchain and Mobile Edge Computing (MEC) are newly emerging technologies with great potential to revolutionize healthcare. This paper proposes a new decentralized healthcare architecture for distributed Electronic Medical Records (EMRs) sharing among federated hospitals based on blockchain and MEC. Unlike the existing schemes that often rely on a third-party for healthcare management, we focus on a fully decentralized access control solution by using smart contracts that enable EMRs access verification at the edge of the network without requiring any central authority. Moreover, a decentralized interplanetary file system (IPFS) platform is also integrated with smart contracts over the MEC network, which significantly reduces data retrieval latency and enhances security for EMRs sharing. The experimental results and analysis show the superior performance of the proposed scheme over the existing ones in terms of reduced data retrieval latency, enhanced blockchain performance, and security guarantees.
Privacy-Preserving Machine Learning Training in IoT Aggregation Scenarios

In developing smart city, the growing popularity of machine learning (ML) that appreciates high-quality training data sets generated from diverse Internet-of-Things (IoT) devices raises natural questions about the privacy guarantees that can be provided in such settings. Privacy-preserving ML training in an aggregation scenario enables a model demander to securely train ML models with the sensitive IoT data gathered from IoT devices. The existing solutions are generally server aided, cannot deal with the collusion threat between the servers or between the servers and data owners, and do not match the delicate environments of IoT. We propose a privacy-preserving ML training framework named Heda that consists of a library of building blocks based on partial homomorphic encryption, which enables constructing multiple privacy-preserving ML training protocols for the aggregation scenario without the assistance of untrusted servers, and defending the security under collusion situations. Rigorous security analysis demonstrates the proposed protocols can protect the privacy of each participant in the honest-but-curious model and guarantee the security under most collusion situations. Extensive experiments validate the efficiency of Heda, which achieves privacy-preserving ML training without losing the model accuracy.