Reviewerly/exHarmony huggingface dataset:
Hypergraph video pedestrian re-identification based on posture structure relationship and action constraints

Discriminative feature learning is critical for pedestrian re-identification. Previous part-based methods mainly focus on the region of specific predefined semantics to learn local representations, ignoring the influence of posture changes, and the learning efficiency and robustness in complex scenes are poor. In this paper, a hypergraph video pedestrian re-identification method based on posture structure relationships and action constraint(PA-HVPReid) is proposed, which aims to make full use of pedestrian walking postures to obtain more discriminative features. The pose structure relationship feature solves the problem that the pooling operation destroys the feature structure relationship. This paper uses a graph convolution network (GCN) to preserve the structure relationship presented by the image feature map. The input of the GCN is the region at the joint points of the pedestrian to be detected, and the output is the color feature that retains the structural relationship. The structural relationship hypergraph is formed according to the structural relationship between the joint point regions. The action hypergraph can be constructed by constraining the action information between the joint point regions. The saliency score of the joint point region is calculated from the posture structure hypergraph and the action hypergraph. We convert the saliency score into a probability distribution problem and propose a relative entropy loss function based on regional saliency to measure the similarity of the two probability distributions. Experimental results show that the performance of our method is better than the existing method on three data sets.
Towards multi-operation image anti-forensics with generative adversarial networks

How to conceal or eliminate the traces left by multiple manipulating operations such as JPEG compression and median filtering successively, known as multi-operation anti-forensics, remains challenging in the field of multimedia security. The existing anti-forensic researches concentrate on concealing or eliminating the traces left by a particular manipulation, referred as single-operation anti-forensics. In this work, we investigate multi-operation anti-forensics with generative adversarial networks (GANs). The generator network is trained to automatically learn the visual and statistical features of the original images by applying appropriate loss functions in the process of optimization. The idea of this work is based on the observation that the single-operation anti-forensic models can be used and extended for multi-operation anti-forensics. We then propose and compare two multi-operation anti-forensic strategies: I) directly using multiple optimized generative models, which are trained for single-operation anti-forensics respectively; II) training the whole GANs to obtain the optimized generative models of multi-operation anti-forensics. The experimental results demonstrate that both strategies can deceive the existing forensic methods without loss of image quality, but the image quality of strategy II is better than that of strategy I which is characterized with the modularity.
Synthetic Sample Selection via Reinforcement Learning

Synthesizing realistic medical images provides a feasible solution to the shortage of training data in deep learning based medical image recognition systems. However, the quality control of synthetic images for data augmentation purposes is under-investigated, and some of the generated images are not realistic and may contain misleading features that distort data distribution when mixed with real images. Thus, the effectiveness of those synthetic images in medical image recognition systems cannot be guaranteed when they are being added randomly without quality assurance. In this work, we propose a reinforcement learning (RL) based synthetic sample selection method that learns to choose synthetic images containing reliable and informative features. A transformer based controller is trained via proximal policy optimization (PPO) using the validation classification accuracy as the reward. The selected images are mixed with the original training data for improved training of image recognition systems. To validate our method, we take the pathology image recognition as an example and conduct extensive experiments on two histopathology image datasets. In experiments on a cervical dataset and a lymph node dataset, the image classification performance is improved by \(8.1\%\) and \(2.3\%\), respectively, when utilizing high-quality synthetic images selected by our RL framework. Our proposed synthetic sample selection method is general and has great potential to boost the performance of various medical image recognition systems given limited annotation.
Dynamic Memory to Alleviate Catastrophic Forgetting in Continuous Learning Settings

In medical imaging, technical progress or changes in diagnostic procedures lead to a continuous change in image appearance. Scanner manufacturer, reconstruction kernel, dose, other protocol specific settings or administering of contrast agents are examples that influence image content independent of the scanned biology. Such domain and task shifts limit the applicability of machine learning algorithms in the clinical routine by rendering models obsolete over time. Here, we address the problem of data shifts in a continuous learning scenario by adapting a model to unseen variations in the source domain while counteracting catastrophic forgetting effects. Our method uses a dynamic memory to facilitate rehearsal of a diverse training data subset to mitigate forgetting. We evaluated our approach on routine clinical CT data obtained with two different scanner protocols and synthetic classification tasks. Experiments show that dynamic memory counters catastrophic forgetting in a setting with multiple data shifts without the necessity for explicit knowledge about when these shifts occur.
Six-Card Finite-Runtime XOR Protocol with Only Random Cut

Executing a card-based cryptographic protocol is an attractive way to perform secure multiparty computation (MPC) with a deck of physical cards. Crèpeau and Kilian at CRYPTO 1993 proposed card-based AND and XOR protocols that can deal with a logical conjunction and exclusive-or of variables. Their protocols use a familiar shuffling action called a random cut that can be easily implemented by humans, while the numbers of required cards and shuffles are not small enough to be efficient and the protocols are Las Vegas algorithms, i.e., they are not finite-runtime. Several researchers have improved upon the protocols for a quarter of a century. Eventually, there are many AND protocols in the literature, some of which are quite efficient and practical to execute. By contrast, there are only three XOR protocols including the Crèpeau--Kilian protocol mentioned above. In this paper, we design an efficient XOR protocol using only random cuts; the numbers of required cards and shuffles are six and two (which is fixed), respectively. Our proposed XOR protocol is the first construction of a finite-runtime XOR protocol if we restrict ourselves to use only random cuts.
Secure Architecture for Inter-Healthcare Electronic Health Records Exchange

The increase in cyberattacks against the healthcare system, notably Electronic Health Records (EHRs) breaches, has cost the healthcare providers more in recent years. This situation is predicted to increase in the coming years as the healthcare systems are proposing a consortium EHRs repository. Due to this reason, it is crucial to deploy solutions that can ensure the security of shared health records. More specifically, maintaining the integrity and consistency of shared EHRs becomes pertinent. In this on-going research, we propose a blockchain-based solution that facilitates a scalable and secured inter-healthcare EHRs exchange. These healthcare systems maintain their records on individual private blockchain networks, and the blockchains interact to exchange patient health history based on request. The proposed solution verifies the integrity and consistency of requests and replies from other healthcare systems. It presents them in a standard format that can be easily understood by different healthcare nodes. The verification steps guard against malicious activities on both stored and in transit EHRs from insider and outsider threat actors. We evaluate the security analysis against frequently encounter outsider and insider threats within a healthcare system. The preliminary result shows that the architecture can detect and prevent threat actors from uploading compromising EHRs into the network and prevents unauthorized retrieval of patient's information.
High PE Utilization CNN Accelerator with Channel Fusion Supporting Pattern-Compressed Sparse Neural Networks

Recently CNN-based methods have made remarkable progress in broad fields. Both network pruning algorithms and hardware accelerators have been introduced to accelerate CNN. However, existing pruning algorithms have not fully studied the pattern pruning method, and current index storage scheme of sparse CNN is not efficient. Furthermore, the performance of existing accelerators suffers from no-load PEs on sparse networks. This work proposes a software-hardware co-design to address these problems. The software includes an ADMM-based method which compresses the patterns of convolution kernels with acceptable accuracy loss, and a Huffman encoding method which reduces index storage overhead. The hardware is a fusion-enabled systolic architecture, which can reduce PEs' no-load rate and improve performance by supporting the channel fusion. On CIFAR-10, this work achieves 5.63× index storage reduction with 2-7 patterns among different layers with 0.87% top-1 accuracy loss. Compared with the state-of-art accelerator, this work achieves 1.54×-1.79× performance and 25%-34% reduction of no-load rate with reasonable area and power overheads.
DeepQoE: Real-time Measurement of Video QoE from Encrypted Traffic with Deep Learning

With the dramatic increase of video traffic on the Internet, video quality of experience (QoE) measurement becomes even more important, which provides network operators with an insight into the quality of their video delivery services. The widespread adoption of end-to-end encryption protocols such as SSL/TLS, however, sets a barrier to QoE monitoring as the most valuable indicators in cleartext traffic are no longer available after encryption. Existing studies on video QoE measurement in encrypted traffic support only coarse-grained QoE metrics or suffer from low accuracy. In this paper, we propose DeepQoE, a new approach that enables real-time video QoE measurement from encrypted traffic. We summarize critical fine-grained QoE metrics, including startup delay, rebuffering, and video resolutions. In order to achieve accurate and real-time inference of these metrics, we build DeepQoE by employing Convolutional Neural Networks (CNNs) with a sophisticated input and architecture design. More specifically, DeepQoE only leverages packet Round-Trip Time (RTT) in upstream traffic as its input. Evaluation results with real-world datasets collected from two popular content providers (i.e., YouTube and Bilibili) show that DeepQoE can improve QoE measurement accuracy by up to 22% over the state-of-the-art methods.
Fuzzy factorization machine

Rational and accurate classification cannot be achieved without considering both the historical information and domain knowledge. We propose fuzzy factorization machine (fuzzy FM) to integrate fuzzy set theory and factorization machine techniques for knowledge-enhanced classification. Each instance is assigned a membership through experts' estimations, and the instance's contribution to the objective function is weighted by its membership instead of the equal penalty in the standard FM. By adopting differentiated weighting strategies, we propose two variants of fuzzy FM: unilaterally weighted fuzzy FM (UFFM) and bilaterally weighted fuzzy FM (BFFM). In BFFM, each instance may not be fully assigned to one of two classes for better classification of imbalanced data, while in UFFM, each instance can only be assigned to one class. A set of membership generation approaches is summarized to quantify experts' prior estimations. We introduce solving methods based on stochastic gradient descent for UFFM and BFFM. Experiments on real credit datasets demonstrate that the proposed fuzzy FM models can yield better rational classification than previous baselines (including the standard FM). The proposed fuzzy FM is a generic machine learning framework that can be applied to various rational classification tasks.
UXNet: Searching Multi-level Feature Aggregation for 3D Medical Image Segmentation

Aggregating multi-level feature representation plays a critical role in achieving robust volumetric medical image segmentation, which is important for the auxiliary diagnosis and treatment. Unlike the recent neural architecture search (NAS) methods that typically searched the optimal operators in each network layer, but missed a good strategy to search for feature aggregations, this paper proposes a novel NAS method for 3D medical image segmentation, named UXNet, which searches both the scale-wise feature aggregation strategies as well as the block-wise operators in the encoder-decoder network. UXNet has several appealing benefits. (1) It significantly improves flexibility of the classical UNet architecture, which only aggregates feature representations of encoder and decoder in equivalent resolution. (2) A continuous relaxation of UXNet is carefully designed, enabling its searching scheme performed in an efficient differentiable manner. (3) Extensive experiments demonstrate the effectiveness of UXNet compared with recent NAS methods for medical image segmentation. The architecture discovered by UXNet outperforms existing state-of-the-art models in terms of Dice on several public 3D medical image segmentation benchmarks, especially for the boundary locations and tiny tissues. The searching computational complexity of UXNet is cheap, enabling to search a network with best performance less than 1.5 days on two TitanXP GPUs.
A Risk-based Path Planning Strategy to Compute Optimum Risk Path for Unmanned Aircraft Systems over Populated Areas

The large diffusion of Unmanned Aircraft Systems (UAS) requires a suitable strategy to design safe flight missions. In this paper, we propose a novel path planning strategy to compute optimum risk path for UAS over populated areas. The proposed strategy is based on a variant of the RRT* (Rapidly-exploring Random Tree "Star") algorithm, performing a risk assessment during the path planning phase. Like other RRT-based algorithms, the proposed path planning explores the state space by constructing a graph. Each time a new node is added to the graph, the algorithm estimates the risk level involved by the new node, evaluating the flight direction and velocity of the UAS placed in the analyzed node. The risk level quantifies the risk of flying over a specific location and it is defined using a probabilistic risk assessment approach taking into account the drone parameters and environmental characteristics. Then, the proposed algorithm computes an asymptotically optimal path by minimizing the overall risk and flight time. Simulation results in realistic environments corroborate the proposed approach proving how the proposed risk-based path planning is able to compute an effective and safe path in urban areas.
Learning Semantic Concepts and Temporal Alignment for Narrated Video Procedural Captioning

Video captioning is a fundamental task for visual understanding. Previous works employ end-to-end networks to learn from the low-level vision feature and generate descriptive captions, which are hard to recognize fine-grained objects and lacks the understanding of crucial semantic concepts. According to DPC [19], these concepts generally present in the narrative transcripts of the instructional videos. The incorporation of transcript and video can improve the captioning performance. However, DPC directly concatenates the embedding of transcript with video features, which is incapable of fusing language and vision features effectively and leads to the temporal mis-alignment between transcript and video. This motivates us to 1) learn the semantic concepts explicitly and 2) design a temporal alignment mechanism to better align the video and transcript for the captioning task. In this paper, we start with an encoder-decoder backbone using transformer models. Firstly, we design a semantic concept prediction module as a multi-task to train the encoder in a supervised way. Then, we develop an attention based cross-modality temporal alignment method that combines the sequential video frames and transcript sentences. Finally, we adopt a copy mechanism to enable the decoder(generation) module to copy important concepts from source transcript directly. The extensive experimental results demonstrate the effectiveness of our model, which achieves state-of-the-art results on YouCookII dataset.
Learning from the Past: Meta-Continual Learning with Knowledge Embedding for Jointly Sketch, Cartoon, and Caricature Face Recognition

This paper deals with a challenging task of learning from different modalities by tackling the difficulty problem of jointly face recognition between abstract-like sketches, cartoons, caricatures and real-life photographs. Due to the significant variations in the abstract faces, building vision models for recognizing data from these modalities is an extremely challenging. We propose a novel framework termed as Meta-Continual Learning with Knowledge Embedding to address the task of jointly sketch, cartoon, and caricature face recognition. In particular, we firstly present a deep relational network to capture and memorize the relation among different samples. Secondly, we present the construction of our knowledge graph that relates image with the label as the guidance of our meta-learner. We then design a knowledge embedding mechanism to incorporate the knowledge representation into our network. Thirdly, to mitigate catastrophic forgetting, we use a meta-continual model that updates our ensemble model and improves its prediction accuracy. With this meta-continual model, our network can learn from its past. The final classification is derived from our network by learning to compare the features of samples. Experimental results demonstrate that our approach achieves significantly higher performance compared with other state-of-the-art approaches.
Asymmetric Deep Hashing for Efficient Hash Code Compression

Benefiting from recent advances in deep learning, deep hashing methods have achieved promising performance in large-scale image retrieval. To improve storage and computational efficiency, existing hash codes need to be compressed accordingly. However, previous deep hashing methods have to retrain their models and then regenerate the whole database codes using the new models when code length changes, which is time consuming especially for large image databases. In this paper, we propose a novel deep hashing method, called Code Compression oriented Deep Hashing (CCDH), for efficiently compressing hash codes. CCDH learns deep hash functions for query images, while learning a one-hidden-layer Variational Autoencoder (VAE) from existing hash codes. With such asymmetric design, CCDH can efficiently compress database codes only using the learned encoder of VAE. Furthermore, CCDH is flexible enough to be used with a variety of deep hashing methods. Extensive experiments on three widely used image retrieval benchmarks demonstrate that CCDH can significantly reduce the cost for compressing database codes when code length changes while keeping the state-of-the-art retrieval accuracy.
SAREF4INMA: A SAREF extension for the industry and manufacturing domain

The IoT landscape is characterized by a fragmentation of standards, platforms and technologies, often scattered among different vertical domains. To prevent the market to continue to be fragmented and power-less, a protocol-independent semantic layer can serve as enabler of interoperability among the various smart devices from different manufacturers that co-exist in a specific industry domain, but also across different domains. To that end, the SAREF ontology was created in 2015 with the intention to interconnect data, enabling the communication between IoT devices that use different protocols and standards. A number of industrial sectors consequently expressed their interest to extend SAREF into their domains in order to fill the gaps of the semantics not yet covered by their communication protocols. Therefore, the SAREF4INMA ontology was recently created to extend SAREF for describing the Smart Industry &amp; Manufacturing domain. SAREF4INMA is based on several standards and IoT initiatives, as well as on real use cases, and includes classes, properties and instances specifically created to cover the industry and manufacturing domain. This work describes the approach followed to develop this ontology, specifies its requirements and also includes a practical example of how to use it.
Dual-view Attention Networks for Single Image Super-Resolution

One non-negligible flaw of the convolutional neural networks (CNNs) based single image super-resolution (SISR) models is that most of them are not able to restore high-resolution (HR) images containing sufficient high-frequency information. Worse still, as the depth of CNNs increases, the training easily suffers from the vanishing gradients. These problems hinder the effectiveness of CNNs in SISR. In this paper, we propose the Dual-view Attention Networks to alleviate these problems for SISR. Specifically, we propose the local aware (LA) and global aware (GA) attentions to deal with LR features in unequal manners, which can highlight the high-frequency components and discriminate each feature from LR images in the local and global views, respectively. Furthermore, the local attentive residual-dense (LARD) block that combines the LA attention with multiple residual and dense connections is proposed to fit a deeper yet easy to train architecture. The experimental results verified the effectiveness of our model compared with other state-of-the-art methods.