Reviewerly/exHarmony huggingface dataset:
LwTE: Light-Weight Transcoding at the Edge

Due to the growing demand for video streaming services, providers have to deal with increasing resource requirements for increasingly heterogeneous environments. To mitigate this problem, many works have been proposed which aim to ( i) improve cloud/edge caching efficiency, (ii) use computation power available in the cloud/edge for on-the-fly transcoding, and (iii) optimize the trade-off among various cost parameters, e.g., storage, computation, and bandwidth. In this paper, we propose LwTE, a novel Light- weight Transcoding approach at the Edge, in the context of HTTP Adaptive Streaming (HAS). During the encoding process of a video segment at the origin side, computationally intense search processes are going on. The main idea of LwTE is to store the optimal results of these search processes as metadata for each video bitrate and reuse them at the edge servers to reduce the required time and computational resources for on-the-fly transcoding. LwTE enables us to store only the highest bitrate plus corresponding metadata (of very small size) for unpopular video segments/bitrates. In this way, in addition to the significant reduction in bandwidth and storage consumption, the required time for on-the-fly transcoding of a requested segment is remarkably decreased by utilizing its corresponding metadata; unnecessary search processes are avoided. Popular video segments/bitrates are being stored. We investigate our approach for Video-on-Demand (VoD) streaming services by optimizing storage and computation (transcoding) costs at the edge servers and then compare it to conventional methods (store all bitrates, partial transcoding). The results indicate that our approach reduces the transcoding time by at least 80% and decreases the aforementioned costs by 12% to 70% compared to the state-of-the-art approaches.
Differential Privacy Meets Federated Learning Under Communication Constraints

The performance of federated learning systems is bottlenecked by communication costs and training variance. The communication overhead problem is usually addressed by three communication-reduction techniques, namely, model compression, partial device participation, and periodic aggregation, at the cost of increased training variance. Different from traditional distributed learning systems, federated learning suffers from data heterogeneity (since the devices sample their data from possibly different distributions), which induces additional variance among devices during training. Various variance-reduced training algorithms have been introduced to combat the effects of data heterogeneity, while they usually cost additional communication resources to deliver necessary control information. Additionally, data privacy remains a critical issue in FL and, thus, there have been attempts at bringing Differential Privacy to this framework as a mediator between utility and privacy requirements. This article investigates the tradeoffs between communication costs and training variance under a resource-constrained federated system theoretically and experimentally, and studies how communication reduction techniques interplay in a differentially private setting. The results provide important insights into designing practical privacy-aware federated learning systems.
The Dynamic Analysis of a Novel Reconfigurable Cubic Chaotic Map and Its Application in Finite Field

Dynamic degradation occurs when chaotic systems are implemented on digital devices, which seriously threatens the security of chaos-based pseudorandom sequence generators. The chaotic degradation shows complex periodic behavior, which is often ignored by designers and seldom analyzed in theory. Not knowing the exact period of the output sequence is the key problem that affects the application of chaos-based pseudorandom sequence generators. In this paper, two cubic chaotic maps are combined, which have symmetry and reconfigurable form in the digital circuit. The dynamic behavior of the cubic chaotic map and the corresponding digital cubic chaotic map are analyzed respectively, and the reasons for the complex period and weak randomness of output sequences are studied. On this basis, the digital cubic chaotic map is optimized, and the complex periodic behavior is improved. In addition, a reconfigurable pseudorandom sequence generator based on the digital cubic chaotic map is constructed from the point of saving consumption of logical resources. Through theoretical and numerical analysis, the pseudorandom sequence generator solves the complex period and weak randomness of the cubic chaotic map after digitization and makes the output sequence have better performance and less resource consumption, which lays the foundation for applying it to the field of secure communication.
Deep Graph Metric Learning for Weakly Supervised Person Re-Identification

In conventional person re-identification (re-id), the images used for model training in the training probe set and training gallery set are all assumed to be instance-level samples that are manually labeled from raw surveillance video (likely with the assistance of detection) in a frame-by-frame manner. This labeling across multiple non-overlapping camera views from raw video surveillance is expensive and time consuming. To overcome these issues, we consider a weakly supervised person re-id modeling that aims to find the raw video clips where a given target person appears. In our weakly supervised setting, during training, given a sample of a person captured in one camera view, our weakly supervised approach aims to train a re-id model without further instance-level labeling for this person in another camera view. The weak setting refers to matching a target person with an untrimmed gallery video where we only know that the identity appears in the video without the requirement of annotating the identity in any frame of the video during the training procedure. The weakly supervised person re-id is challenging since it not only suffers from the difficulties occurring in conventional person re-id (e.g., visual ambiguity and appearance variations caused by occlusions, pose variations, background clutter, etc.), but more importantly, is also challenged by weakly supervised information because the instance-level labels and the ground-truth locations for person instances (i.e., the ground-truth bounding boxes of person instances) are absent. To solve the weakly supervised person re-id problem, we develop deep graph metric learning (DGML). On the one hand, DGML measures the consistency between intra-video spatial graphs of consecutive frames, where the spatial graph captures neighborhood relationship about the detected person instances in each frame. On the other hand, DGML distinguishes the inter-video spatial graphs captured from different camera views at different sites simultaneously. To further explicitly embed weak supervision into the DGML and solve the weakly supervised person re-id problem, we introduce weakly supervised regularization (WSR), which utilizes multiple weak video-level labels to learn discriminative features by means of a weak identity loss and a cross-video alignment loss. We conduct extensive experiments to demonstrate the feasibility of the weakly supervised person re-id approach and its special cases (e.g., its bag-to-bag extension) and show that the proposed DGML is effective.
Tackling Virtual and Real Concept Drifts: An Adaptive Gaussian Mixture Model Approach

Real-world applications have been dealing with large amounts of data that arrive over time and generally present changes in their underlying joint probability distribution, i.e., concept drift.Concept drift can be subdivided into two types: virtual drift, which affects the unconditional probability distribution p(x), and real drift, which affects the conditional probability distribution p(y|x).Existing works focuses on real drift.However, strategies to cope with real drift may not be the best suited for dealing with virtual drift, since the real class boundaries remain unchanged.We provide the first in depth analysis of the differences between the impact of virtual and real drifts on classifiers' suitability.We propose an approach to handle both drifts called On-line Gaussian Mixture Model With Noise Filter For Handling Virtual and Real Concept Drifts (OGMMF-VRD).Experiments with seven synthetics and seven real-world datasets show that OGMMF-VRD outperforms other approaches with separate mechanisms to deal with virtual and real drifts.It also has more stable rankings and smaller drops in performance during drifting periods than existing ensemble approaches, thus being more reliable for adoption in practice.
DeepRx MIMO: Convolutional MIMO Detection with Learned Multiplicative Transformations

Recently, deep learning has been proposed as a potential technique for improving the physical layer performance of radio receivers. Despite the large amount of encouraging results, most works have not considered spatial multiplexing in the context of multiple-input and multiple-output (MIMO) receivers. In this paper, we present a deep learning-based MIMO receiver architecture that consists of a ResNet-based convolutional neural network, also known as DeepRx, combined with a so-called transformation layer, all trained together. We propose two novel alternatives for the transformation layer: a maximal ratio combining-based transformation, or a fully learned transformation. The former relies more on expert knowledge, while the latter utilizes learned multiplicative layers. Both proposed transformation layers are shown to clearly outperform the conventional baseline receiver, especially with sparse pilot configurations. To the best of our knowledge, these are some of the first results showing such high performance for a fully learned MIMO receiver.
X-MAN: Explaining multiple sources of anomalies in video

Our objective is to detect anomalies in video while also automatically explaining the reason behind the detector’s response. In a practical sense, explainability is crucial for this task as the required response to an anomaly depends on its nature and severity. However, most leading methods (based on deep neural networks) are not interpretable and hide the decision making process in uninterpretable feature representations. In an effort to tackle this problem we make the following contributions: (1) we show how to build interpretable feature representations suitable for detecting anomalies with state of the art performance, (2) we propose an interpretable probabilistic anomaly detector which can describe the reason behind it’s response using high level concepts, (3) we are the first to directly consider object interactions for anomaly detection and (4) we propose a new task of explaining anomalies and release a large dataset for evaluating methods on this task. Our method competes well with the state of the art on public datasets while also providing anomaly explanation based on objects and their interactions.
REUR: A unified deep framework for signet ring cell detection in low-resolution pathological images

Detecting signet ring cells (SRCs) in pathological images is essential for carcinoma diagnosis. However, it is time consuming for pathologists to detect SRCs manually from pathological images, and the accuracy of detecting them is also relatively low because of their small sizes. Recently, the exploration of deep learning methods in pathology analysis has been widely investigated by researchers. Nevertheless, the automatic detection of SRCs from real pathological images faces two problems. One is that labeled pathological images are insufficient and usually incomplete. The other is that the training data and the real clinical data have a large difference in resolution. Hence, adopting the transfer learning method affects the performance of deep learning methods. To address these two problems, we present a unified framework named REUR [RetinaNet combining USRNet (unfolding super-resolution network) with the RGHMC (revised gradient harmonizing mechanism classification) loss] that can accurately detect SRCs in low-resolution (LR) pathological images. First, the framework with the super-resolution (SR) module can address the difference in resolution between the training data and the real clinical data. Second, the framework with the label correction module can obtain the revised ground-truth labels from noisy examples, which are embedded into the gradient harmonizing mechanism to acquire the RGHMC loss. The results of the numerical experiments showed that the framework can perform better than other one-stage detectors based on the RetinaNet architecture in the high-resolution (HR) noisy dataset. It achieved a kappa value of 0.74 and an accuracy of 0.89 in the test with 27 randomly selected whole slide images (WSIs), and, thus, it can assist pathologists in better analyzing WSIs. The framework provides an essential method in computer-aided diagnosis for medical applications.
Facial emotion recognition using convolutional neural networks

Emotional expressivity has always been a simple job for people, but computer programming is much harder to accomplish. Image emotions may be recognised by recent developments in computer vision and machine learning. In this article, we present a new method to detect face emotion. Use neural networks convolutionary (FERC). The FERC is based on a CNN network of two parts: the first portion removed the backdrop of the image, the second part removed the face vector. The expressional vector (EV) is utilised in the FERC model to detect the fve different kinds of regular facial expressions. The double-level CNN is continuous and the weights and exponent values of the final perception layer vary with each iteration. In that it improves accuracy, FERC varies from widely utilised CNN single-level technology. Moreover, EV generation prevents the development of a number of issues before a new background removal process is used (for example distance from the camera).
DiaKG: An Annotated Diabetes Dataset for Medical Knowledge Graph Construction

Knowledge Graph has been proven effective in modeling structured information and conceptual knowledge, especially in the medical domain. However, the lack of high-quality annotated corpora remains a crucial problem for advancing the research and applications on this task. In order to accelerate the research for domain-specific knowledge graphs in the medical domain, we introduce DiaKG, a high-quality Chinese dataset for Diabetes knowledge graph, which contains 22,050 entities and 6,890 relations in total. We implement recent typical methods for Named Entity Recognition and Relation Extraction as a benchmark to evaluate the proposed dataset thoroughly. Empirical results show that the DiaKG is challenging for most existing methods and further analysis is conducted to discuss future research direction for improvements. We hope the release of this dataset can assist the construction of diabetes knowledge graphs and facilitate AI-based applications.
A Transparent and Traceable Coverage Analysis Model for Vaccine Supply-Chain Using Blockchain Technology

Tracing the vaccines from the manufacturing labs till the health centers administering doses to child is a cumbersome process with the involvement of large number of stakeholders and with multiple levels of book-keeping mechanisms. This work primarily focuses on the problem of traceability of immunization vaccines from manufacturing facility, through the distribution mechanisms in cold storage network and the vaccine handling/administration facilities, till the end consumer in the Indian context. The proposed research named, Immunochain is a state-of-the-art big data and blockchain technologies powered, mobile/web-enabled vaccine traceability solution for immunization programs in India. It is vendor neutral, scalable, replicable, and reusable across different immunization programs in any geographical context. Experiments and pilot testing on Indian geography indicates that the proposed approach outperforms some of the existing approaches in vaccine traceability and record management systems.
Learning Context-Based Nonlocal Entropy Modeling for Image Compression

The entropy of the codes usually serves as the rate loss in the recent learned lossy image compression methods. Precise estimation of the probabilistic distribution of the codes plays a vital role in reducing the entropy and boosting the joint rate-distortion performance. However, existing deep learning based entropy models generally assume the latent codes are statistically independent or depend on some side information or local context, which fails to take the global similarity within the context into account and thus hinders the accurate entropy estimation. To address this issue, we propose a special nonlocal operation for context modeling by employing the global similarity within the context. Specifically, due to the constraint of context, nonlocal operation is incalculable in context modeling. We exploit the relationship between the code maps produced by deep neural networks and introduce the proxy similarity functions as a workaround. Then, we combine the local and the global context via a nonlocal attention block and employ it in masked convolutional networks for entropy modeling. Taking the consideration that the width of the transforms is essential in training low distortion models, we finally produce a U-net block in the transforms to increase the width with manageable memory consumption and time complexity. Experiments on Kodak and Tecnick datasets demonstrate the priority of the proposed context-based nonlocal attention block in entropy modeling and the U-net block in low distortion situations. On the whole, our model performs favorably against the existing image compression standards and recent deep image compression models.
Point Cloud Hand–Object Segmentation Using Multimodal Imaging with Thermal and Color Data for Safe Robotic Object Handover

This paper presents an application of neural networks operating on multimodal 3D data (3D point cloud, RGB, thermal) to effectively and precisely segment human hands and objects held in hand to realize a safe human-robot object handover. We discuss the problems encountered in building a multimodal sensor system, while the focus is on the calibration and alignment of a set of cameras including RGB, thermal, and NIR cameras. We propose the use of a copper-plastic chessboard calibration target with an internal active light source (near-infrared and visible light). By brief heating, the calibration target could be simultaneously and legibly captured by all cameras. Based on the multimodal dataset captured by our sensor system, PointNet, PointNet++, and RandLA-Net are utilized to verify the effectiveness of applying multimodal point cloud data for hand-object segmentation. These networks were trained on various data modes (XYZ, XYZ-T, XYZ-RGB, and XYZ-RGB-T). The experimental results show a significant improvement in the segmentation performance of XYZ-RGB-T (mean Intersection over Union: 82.8% by RandLA-Net) compared with the other three modes (77.3% by XYZ-RGB, 35.7% by XYZ-T, 35.7% by XYZ), in which it is worth mentioning that the Intersection over Union for the single class of hand achieves 92.6%.
Transfer Learning and Decision Fusion for Real Time Distortion Classification in Laparoscopic Videos

Laparoscopic surgery is a surgical procedure performed by inserting narrow tubes into the abdomen without making large incisions in the skin. It is done with the aid of a video camera. Laparoscopic videos are affected by various distortions during surgery which lead to loss of visual quality. Identification of these distortions is the primary requisite in automated video enhancement systems used to classify the distortions correctly and accordingly select the proper algorithm to enhance video quality. In addition to high accuracy, the speed of distortion classification should be high, and the system must consider real-time conditions. This paper aims to address the issues faced by similar methods by developing a fast and accurate deep learning model for distortion classification. The dataset proposed by the ICIP2020 conference challenge was used for training and evaluation of the proposed method. This challenging dataset contains videos that have five types of distortions such as noise, smoke, uneven illumination, defocus blur, and motion blur with four levels of intensity. This paper discusses the proposed solution which received the first prize in the ICIP2020 challenge. The solution utilized a transfer learning approach to transfer representation from the domain of natural images to the domain of laparoscopic videos. We used a pre-trained ResNet50 convolutional neural network (CNN) to extract informative features that were mapped by support vector machine (SVM) classifiers to various distortion categories. In this work, the problem of multiple distortions in the same video was formulated as a multi-label distortion classification problem. The approach of transfer learning with decision fusion was applied and was found to outperform other solutions in terms of accuracy (83%), F1 score of a single distortion (94.7%), and F1 score of single and multiple distortions (94.9%). In addition, the proposed solution can run in real time with an inference speed of 20 frames per second (FPS).
Higher-order Interaction Goes Neural: A Substructure Assembling Graph Attention Network for Graph Classification

Graph classification has been widely used for knowledge discovery in numerous practical application scenarios, such as social networks and protein-protein interaction networks. Recently, Graph Neural Networks (GNNs), which generalize deep neural networks to graph-structured data, have drawn considerable attention and achieved state-of-the-art performance in graph classification. However, existing GNN models mainly focus on capturing the information of immediate or first-order neighboring nodes within a single layer. The graph substructure and substructure interaction, that play an important role in learning graph representations, are usually overlooked. In this paper, we propose a Substructure Assembling Graph Attention Network (SA-GAT) to extract graph features and improve the performance of graph classification. SA-GAT is able to fully explore higher-order substructure information hidden in graphs by a core module called Substructure Interaction Attention (SIA). Theoretically, we have also proved that SA-GAT satisfies the graph isomorphism theory of graph neural network design, which is that the network should map isomorphic graphs to the same representation and output the same prediction. Extensive experimental results on multiple real-world graph classification datasets demonstrate that the proposed SA-GAT outperforms the state-of-the-art methods including graph kernels and graph neural networks.