Reviewerly/exHarmony huggingface dataset:
Deep Outlier Handling for Image Deblurring

Outlier handling has attracted considerable attention recently but remains challenging for image deblurring. Existing approaches mainly depend on iterative outlier detection steps to explicitly or implicitly reduce the influence of outliers on image deblurring. However, these outlier detection steps usually involve heuristic operations and iterative optimization processes, which are complex and time-consuming. In contrast, we propose to learn a deep convolutional neural network to directly estimate the confidence map, which can identify reliable inliers and outliers from the blurred image and thus facilitates the following deblurring process. We analyze that the proposed algorithm incorporated with the learned confidence map is effective in handling outliers and does not require ad-hoc outlier detection steps which are critical to existing outlier handling methods. Compared to existing approaches, the proposed algorithm is more efficient and can be applied to both non-blind and blind image deblurring. Extensive experimental results demonstrate that the proposed algorithm performs favorably against state-of-the-art methods in terms of accuracy and efficiency.
WITHDRAWN: Task scheduling to reduce energy consumption and makespan of cloud computing using NSGA-II

Cloud service usage has significantly increased due to ease in access, improved performance, and low acquisition costs. All the cloud users plan to get their jobs done quickly. Still, cloud providers are looking for ways to reduce energy costs, one of the highest prices in a cloud service environment. This is achieved by reducing the number of machines used to get the users' job done. Consecutively, the makespan on the active machines increases, which often leads to customer dissatisfaction. This article proposes a multi-objective optimization model to minimize energy consumption and makespan for efficient task scheduling. This article additionally offers a Non-dominated Sorting Genetic Algorithm (NSGA) based algorithm to solve the optimization multi-objective task allocation problem. The performance of the proposed model and algorithms has been analyzed over various comprehensive experiments. The experimental results analysis highlights that the proposed model and algorithm reduces the makespan and energy consumption significantly compared to other traditional algorithms.
Anonymous Authenticated Key Agreement and Group Proof Protocol for Wearable Computing

Wearable computing has been used in a wide range of applications. But wearable computing often suffers from various security and privacy issues. To solve these issues, many effective authentication schemes have been proposed. However, most of the existing schemes are vulnerable to various known attacks (such as desynchronization attack, privileged-insider attack, and anonymity attack), or require high computation and communication costs, and are not suitable for resource-constrained wearable devices, or simultaneous verification of multiple wearable devices is not supported. Therefore, in this paper, we propose a new anonymous authentication and group proof protocol for wearable computing, which achieves mutual authentication between the wearable device and user and between user and cloud server, and generates a group proof for multiple wearable devices. Further, we extend the Real-Or-Random (ROR) model to support anonymity and group proof, and formally prove that the proposed scheme is provably secure under the extended security model. In addition, the informal security analysis is demonstrated that the proposed scheme is more resilient against known attacks. Finally, compared with some existing schemes, the proposed scheme offers more functionality features and requires less communication and computation costs.
An Online Offline Framework for Anomaly Scoring and Detecting New Traffic in Network Streams

Network data constantly evolves with new network applications and protocols. There is a need for robust techniques to detect anomalous behaviour. Offline models trained with static data lose validity when new variants of traffic emerge. They require retraining but the need for ground truth and lengthy training times make this task challenging. Meanwhile, online models which detect outliers in streaming data are susceptible to the curse of dimensionality and natural variability. Today’s anomalies may be tomorrow’s new traffic and existing methods do not provide a way to differentiate between them. We propose a framework that makes the most of both approaches: an offline deep learning model extracts features of normal traffic and provides a bias for an online outlier detection model to select data for training. The online model retains its previously learnt knowledge and retrains itself with new data. Online thresholds are updated in a drifting manner and the Mann-Whitney U test is incorporated to prevent inaccurate updates. We perform analysis on the scores, develop heuristics to detect new traffic and evaluate using three deep learning models and four outlier detection methods on the UNSW-NB15 and CTU-13 datasets. The framework improves upon any individual offline or online models in isolation.
Caps-OWKG: a capsule network model for open-world knowledge graph

Abstract Knowledge graphs are typical multi-relational structures, which is consisted of many entities and relations. Nonetheless, existing knowledge graphs are still sparse and far from being complete. To refine the knowledge graphs, representation learning is utilized to embed entities and relations into low-dimensional spaces. Many existing knowledge graphs embedding models focus on learning latent features in close-world assumption but omit the changeable of each knowledge graph.In this paper, we propose a knowledge graph representation learning model, called Caps-OWKG, which leverages the capsule network to capture the both known and unknown triplets features in open-world knowledge graph. It combines the descriptive text and knowledge graph to get descriptive embedding and structural embedding, simultaneously. Then, the both above embeddings are used to calculate the probability of triplet authenticity. We verify the performance of Caps-OWKG on link prediction task with two common datasets FB15k-237-OWE and DBPedia50k. The experimental results are better than other baselines, and achieve the state-of-the-art performance.
EAGLE-Eye: Extreme-pose Action Grader using detaiL bird’s-Eye view

Measuring the quality of a sports action entails attending to the execution of the short-term components as well as overall impression of the whole program. In this assessment, both appearance clues and pose dynamics features should be involved. Current approaches often treat a sports routine as a simple fine-grained action, while taking little heed of its complex temporal structure. Besides, they rely solely on either appearance or pose features to score the performance. In this paper, we present JCA and ADA blocks that are responsible for reasoning about the coordination among the joints and appearance dynamics throughout the performance. We build our two-stream network upon the separate stack of these blocks. The early blocks capture the fine-grained temporal dependencies while the last ones reason about the long-term coarse-grained relations. We further introduce an annotated dataset of sports images with unusual pose configurations to boost the performance of pose estimation in such scenarios. Our experiments show that the proposed method not only outperforms the previous works in short-term action assessment but also is the first to generalize well to minute-long figure-skating scoring.
Towards Precise Intra-camera Supervised Person Re-Identification

Intra-camera supervision (ICS) for person reidentification (Re-ID) assumes that identity labels are independently annotated within each camera view and no inter-camera identity association is labeled. It is a new setting proposed recently to reduce the burden of annotation while expect to maintain desirable Re-ID performance. However, the lack of inter-camera labels makes the ICS Re-ID problem much more challenging than the fully supervised counterpart. By investigating the characteristics of ICS, this paper proposes jointly learned camera-specific non-parametric classifiers, together with a hybrid mining quintuplet loss, to perform intra-camera learning. Then, an inter-camera learning module consisting of a graph-based ID association step and a Re-ID model updating step is conducted. Extensive experiments on three large-scale Re-ID datasets show that our approach outperforms all existing ICS works by a great margin. Our approach performs even comparable to state-of-the-art fully supervised methods in two of the datasets.
A person-centred approach to L2 learners’ informal mobile language learning

Mobile technologies provide opportunities for L2 learners to engage in complex interactions involving a multitude of cognitive, meta-cognitive, and affective factors. Understanding the process of learners’ mobile language learning thus needs holistic approaches that integratively consider learner attributes and their interaction with mobile technologies. In this study, we applied a holistic person-centred approach to examining L2 learners’ self-initiated engagement with mobile learning activities. Data were collected with a questionnaire and follow-up interviews. A cluster analysis performed on the questionnaire data yielded six types of learners with distinctly different learning experiences. We further found that each learner type brought along a distinct package of motivational, emotional, and linguistic interaction, and that the distinct learning experiences of each learner type bore different relationship to learners’ perceived L2 improvement. This study offers theoretical and methodological insights into the complexity and variety inherent in informal mobile language learning. Findings also inform teachers about the design of adaptive and tailored instruction and scaffolding.
Depth Map Refinement for Immersive Video

In this article, we propose a depth map refinement method that increases the quality of immersive video.The proposal highly enhances the inter-view consistency of depth maps (estimated or acquired by any method), crucial for achieving the required fidelity of the virtual view synthesis process.In the described method, only information from depth maps is used, as the use of texture can introduce errors in the refinement, mostly due to inter-view color inconsistencies and noise.In order to evaluate the performance of the proposal and compare it with the state of the art, three experiments were conducted.To test the influence of the refinement on the encoding of immersive video, four sets of depth maps (original, refined with the synthesis-based refinement, a bilateral filter, and with the proposal) were encoded with the MPEG Immersive Video (MIV) encoder.In the second experiment, in order to provide a direct evaluation of the accuracy of depth maps, the Middlebury database comparison was performed.In the third experiment, the temporal consistency of depth maps was assessed by measuring the efficiency of encoding of the virtual views.The experiments showed both a high increase of the virtual view synthesis quality in immersive video applications and higher similarity to ground-truth after the refinement of estimated depth maps.The usefulness of the proposal was appreciated and confirmed by the experts of the ISO/IEC MPEG group for immersive video and the method became the MPEG Reference Software for the depth refinement.The implementation of the method is publicly available for other researchers.
Exploiting Informative Video Segments for Temporal Action Localization

We propose a novel method of exploiting informative video segments by learning segment weights for temporal action localization in untrimmed videos. Informative video segments represent the intrinsic motion and appearance of an action, and thus contribute crucially to action localization. The learned segment weights represent the informativeness of video segments to recognize actions and help infer the boundaries required to temporally localize actions. We build a supervised temporal attention network (STAN) that includes a supervised segment-level attention module to dynamically learn the weights of video segments, and a feature-level attention module to effectively fuse multiple features of segments. Through the cascade of the attention modules, STAN exploits informative video segments and generates descriptive and discriminative video representations. We use a proposal generator and a classifier to estimate the boundaries of actions and classify the classes of actions. Extensive experiments are conducted on two public benchmarks, i.e., THUMOS2014 and ActivityNet1.3. The results demonstrate that our proposed method achieves competitive performance compared with existing state-of-the-art methods. Moreover, compared with the baseline method that treats video segments equally, STAN achieves significant improvements with an increase of the mean average precision from 30.4% to 39.8% on the THUMOS2014 dataset, and from 31.4% to 35.9% on the ActivityNet1.3 dataset, demonstrating the effectiveness of learning informative video segments for temporal action localization.
Real-Time Instance Segmentation of Traffic Videos for Embedded Devices

The paper proposes a novel instance segmentation method for traffic videos devised for deployment on real-time embedded devices. A novel neural network architecture is proposed using a multi-resolution feature extraction backbone and improved network designs for the object detection and instance segmentation branches. A novel post-processing method is introduced to ensure a reduced rate of false detection by evaluating the quality of the output masks. An improved network training procedure is proposed based on a novel label assignment algorithm. An ablation study on speed-vs.-performance trade-off further modifies the two branches and replaces the conventional ResNet-based performance-oriented backbone with a lightweight speed-oriented design. The proposed architectural variations achieve real-time performance when deployed on embedded devices. The experimental results demonstrate that the proposed instance segmentation method for traffic videos outperforms the you only look at coefficients algorithm, the state-of-the-art real-time instance segmentation method. The proposed architecture achieves qualitative results with 31.57 average precision on the COCO dataset, while its speed-oriented variations achieve speeds of up to 66.25 frames per second on the Jetson AGX Xavier module.
CELOF: Effective and fast memory efficient local outlier detection in high-dimensional data streams

Outlier detection is an important and challenging problem in industrial automation, where data are often collected in large amounts but with little labeled information. To realize real-time outlier detection on data streams, many models have been proposed in the academic. However, most existing outlier detection algorithms still have two main limitations: (1) Need a large amount of memory to store data. (2) Poor detection of high-dimensional data in application scenarios. In this paper, we propose a new algorithm, called CELOF which can effectively overcome the two limitations. In CELOF, We first use information entropy to construct a new index weight calculation method, which can distinguish the influencing factors of different indexes and improve the detection accuracy of multi-dimensional data. Next, we designed a new reachable distance factor discrimination method to extract the original data information and then proposed a new strategy for outlier detection, which can greatly reduce the amount of data storage. Finally, the final experiment result shows that the CELOF algorithm has an average improvement of 15% in accuracy compared to the state-of-the-art algorithms, and the CELOF’s running time less than 1% of the original LOF. Additionally, our comprehensive experiments use different real data sets for simulation, and the results show that our algorithm can be widely used in different practical application scenarios without any prior information and data distribution.
Robust Local Structure Visualization for Remote Sensing Image Registration

Image registration is a fundamental and important task in remote sensing. In this article, we focus on feature-based image registration. Existing attempts often require estimating a transformation model or imposing relaxed geometric constraints to establish reliable feature correspondences. However, a parametric model cannot handle image pairs undergoing complex transformations, and relaxed methods discard a lot of structure information and the results are often coarse. To solve the above issues, we propose a local structure visualization descriptor to preserve the original structure information, and cast the feature matching task into an evaluation of the consensus of visual structure under a convolutional neural network. This strategy can effectively measure the similarity of neighborhood structure for mismatch removal. In summary, our method does not depend on a specific transformation model and can process arbitrary remote sensing images (e.g., different deformations, severe outliers, various rotations, and scaling changes). To demonstrate the robustness of our strategy for image registration, extensive experiments on various real remote sensing images for feature matching are conducted and compared against nine state-of-the-art methods, where our method gives the best performances in most scenarios.
Towards Zero-Shot Knowledge Distillation for Natural Language Processing

Knowledge distillation (KD) is a common knowledge transfer algorithm used for model compression across a variety of deep learning based natural language processing (NLP) solutions.In its regular manifestations, KD requires access to the teacher's training data for knowledge transfer to the student network.However, privacy concerns, data regulations and proprietary reasons may prevent access to such data.We present, to the best of our knowledge, the first work on Zero-shot Knowledge Distillation for NLP, where the student learns from the much larger teacher without any task specific data.Our solution combines out-ofdomain data and adversarial training to learn the teacher's output distribution.We investigate six tasks from the GLUE benchmark and demonstrate that we can achieve between 75% and 92% of the teacher's classification score (accuracy or F1) while compressing the model 30 times.
GAN-Based Anomaly Detection In Imbalance Problems

Imbalance pre one of the key issues that affect the performance greatly. Our focus in this work is to address an imbalance problem arising from defect detection in industrial inspections, including the different number of defect and non-defect dataset, the gap of distribution among defect classes, and various sizes of defects. To this end, we adopt the anomaly detection method that is to identify unusual patterns to address such challenging problems. Especially generative adversarial network (GAN) and autoencoder-based approaches have shown to be effective in this field. In this work, (1) we propose a novel GAN-based anomaly detection model which consists of an autoencoder as the generator and two separate discriminators for each of normal and anomaly input; and (2) we also explore a way to effectively optimize our model by proposing new loss functions: Patch loss and Anomaly adversarial loss, and further combining them to jointly train the model. In our experiment, we evaluate our model on conventional benchmark datasets such as MNIST, Fashion MNIST, CIFAR 10/100 data as well as on real-world industrial dataset – smartphone case defects. Finally, experimental results demonstrate the effectiveness of our approach by showing the results of outperforming the current State-Of-The-Art approaches in terms of the average area under the ROC curve (AUROC).
Variational Prototype Inference for Few-Shot Semantic Segmentation

In this paper, we propose variational prototype inference to address few-shot semantic segmentation in a probabilistic framework.A probabilistic latent variable model infers the distribution of the prototype that is treated as the latent variable.We formulate the optimization as a variational inference problem, which is established with an amortized inference network based on an auto-encoder architecture.The probabilistic modeling of the prototype enhances its generalization ability to handle the inherent uncertainty caused by limited data and the huge intra-class variations of objects.Moreover, it offers a principled way to incorporate the prototype extracted from support images into the prediction of the segmentation maps for query images.We conduct extensive experimental evaluations on three benchmark datasets.Ablation studies show the effectiveness of variational prototype inference for few-shot semantic segmentation by probabilistic modeling.On all three benchmarks, our proposal achieves high segmentation accuracy and surpasses previous methods by considerable margins.
Volumetric reconstruction for combustion diagnostics via transfer learning and semi-supervised learning with limited labels

Volumetric tomography (VT) is a powerful tool for combustion diagnostics due to its capacity in resolving flame structures in three-dimensional (3D). Recently, convolutional neural network (CNN) has been applied to solve the inversion problems of VT, which features an overwhelming advantage over classical iterative methods in terms of computational efficiency. However, a large number of labels have to be prepared for the supervised learning of CNN using iterative methods, compromising its efficiency advantage. Moreover, previous studies were limited to a single dataset and the generalization performance of CNN has not yet been tested. In this work, both transfer learning and semi-supervised learning were employed to construct the CNN networks with limited labels. The comparative studies between them and supervised learning confirmed that a significant improvement in reconstruction accuracy can be achieved even with limited labels. The correlation coefficient between the reconstruction and ground truth is larger than 0.98 for three commonly encountered application scenarios. The training strategies developed in this work are expected to be valuable for all VT modalities as applied to flow/combustion diagnostics.
Real-Time Hair Segmentation Using Mobile-Unet

We described a real-time hair segmentation method based on a fully convolutional network with the basic structure of an encoder–decoder. In one of the traditional computer vision techniques for hair segmentation, the mean shift and watershed methodologies suffer from inaccuracy and slow execution due to multi-step, complex image processing. It is also difficult to execute the process in real-time unless an optimization technique is applied to the partition. To solve this problem, we exploited Mobile-Unet using the U-Net segmentation model, which incorporates the optimization techniques of MobileNetV2. In experiments, hair segmentation accuracy was evaluated by different genders and races, and the average accuracy was 89.9%. By comparing the accuracy and execution speed of our model with those of other models in related studies, we confirmed that the proposed model achieved the same or better performance. As such, the results of hair segmentation can obtain hair information (style, color, length), which has a significant impact on human-robot interaction with people.