Reviewerly/exHarmony huggingface dataset:
VLSI Implementation of Image Compression using TSA Optimized Discrete Wavelet Transform Techniques

The Discrete Wavelet transform is one of the best tools for signal and data analysis, It requires efficient hardware implementation in the real-time applications. The submissions established in the field of imaging necessitates compacted architecture. In DWT discrete sampling is accomplished for the wavelets. In this paper, the VLSI implementation of HAAR wavelet-based image compression is proposed and designed. HAAR wavelet transform is one of the easiest methods for image compression because it has coefficients as either 1 or -1. In this work software alone is used for the compression together with optimizing it with a continuous optimization algorithm and provides a hardware-free architecture with low cost. The VHDL work is carried out in Xilinx Platform and provides a truncated power architecture for a concrete application. The same VHDL architecture can also be instigated in FPGA which will harvest hardware effectual compromising outcomes.
Single image super-resolution with multi-scale information cross-fusion network

The deep convolutional neural networks have achieved significant improvements in terms of effectiveness and efficiency for single image super-resolution. However, as the depth of neural network grows, the information flow is weakened. On the other hand, most of the models adopt a single-stream structure with which integrating complementary information under different receptive fields is difficult. To improve information flow and capture sufficient knowledge for reconstructing high-frequency details, we propose a multi-scale information cross-fusion network (MSICF) in which a sequence of subnetworks is cascaded to infer high resolution features in a coarse-to-fine manner. In each cascaded subnetwork, we stack multiple multi-scale cross (MSC) modules to fuse multi-scale complementary information in an efficient way as well as to improve information flow across the layers. Meanwhile, by introducing residual-feature learning in each stage, the relative information between features at different levels is fully utilized to further boost reconstruction performance. We train the proposed network with cascaded-supervision and then assemble the intermediate predictions of the cascade to achieve high quality image. Extensive quantitative and qualitative evaluations on benchmark datasets illustrate the superiority of our proposed method over state-of-the-art super-resolution methods.
Ice and snow talent training based on construction and analysis of artificial intelligence education informatization teaching model

Based on the analysis of the artificial intelligence education informatization teaching model for the training of ice and snow talents, this article first builds an artificial intelligence education informatization teaching model, reads and organizes a large number of documents such as big data and personalized teaching of ice and snow talent training. It needs to analyze the existing problems and condense the relevant concepts of educational big data, personalized teaching. We personalized teaching systems based on big data and elaborate on the related theoretical basis. It constructed a visual analysis framework for artificial intelligence teaching data, discussed the realization process and mechanism of data visualization of mixed ice and snow talent cultivation from the two dimensions of timeliness and media form, and introduced the mixed data of numerical and text for visual processing methods. We discussed the interactive presentation process of the visualization results. The research explores the impact of artificial intelligence on the elements of ice and snow talent training instruction design one by one and uses the paradigm migration analysis framework to prove that the ice and snow talent training instruction design paradigm in the context of artificial intelligence has produced a migration.
Review of Natural Language Processing in Radiology

Natural language processing (NLP) is an interdisciplinary field, combining linguistics, computer science, and artificial intelligence to enable machines to read and understand human language for meaningful purposes. Recent advancements in deep learning have begun to offer significant improvements in NLP task performance. These techniques have the potential to create new automated tools that could improve clinical workflows and unlock unstructured textual information contained in radiology and clinical reports for the development of radiology and clinical artificial intelligence applications. These applications will combine the appropriate application of classic linguistic and NLP preprocessing techniques, modern NLP techniques, and modern deep learning techniques.
MS-NAS: Multi-scale Neural Architecture Search for Medical Image Segmentation

The recent breakthroughs of Neural Architecture Search (NAS) have motivated various applications in medical image segmentation. However, most existing work either simply rely on hyper-parameter tuning or stick to a fixed network backbone, thereby limiting the underlying search space to identify more efficient architecture. This paper presents a Multi-Scale NAS (MS-NAS) framework that is featured with multi-scale search space from network backbone to cell operation, and multi-scale fusion capability to fuse features with different sizes. To mitigate the computational overhead due to the larger search space, a partial channel connection scheme and a two-step decoding method are utilized to reduce computational overhead while maintaining optimization quality. Experimental results show that on various datasets for segmentation, MS-NAS outperforms the state-of-the-art methods and achieves 0.6–5.4% mIOU and 0.4–3.5% DSC improvements, while the computational resource consumption is reduced by 18.0–24.9%.
Hier R-CNN: Instance-Level Human Parts Detection and A New Benchmark

Detecting human parts at instance-level is an essential prerequisite for the analysis of human keypoints, actions, and attributes. Nonetheless, there is a lack of a large-scale, rich-annotated dataset for human parts detection. We fill in the gap by proposing COCO Human Parts. The proposed dataset is based on the COCO 2017, which is the first instance-level human parts dataset, and contains images of complex scenes and high diversity. For reflecting the diversity of human body in natural scenes, we annotate human parts with (a) location in terms of a bounding-box, (b) various type including face, head, hand, and foot, (c) subordinate relationship between person and human parts, (d) fine-grained classification into right-hand/left-hand and left-foot/right-foot. A lot of higher-level applications and studies can be founded upon COCO Human Parts, such as gesture recognition, face/hand keypoint detection, visual actions, human-object interactions, and virtual reality. There are a total of 268,030 person instances from the 66,808 images, and 2.83 parts per person instance. We provide a statistical analysis of the accuracy of our annotations. In addition, we propose a strong baseline for detecting human parts at instance-level over this dataset in an end-to-end manner, call Hier(archy) R-CNN. It is a simple but effective extension of Mask R-CNN, which can detect human parts of each person instance and predict the subordinate relationship between them. Codes and dataset are publicly available (https://github.com/soeaver/Hier-R-CNN).
CFVMNet

Vehicle re-identification (re-ID) aims to retrieve the image of the same vehicles across multiple cameras. It has attracted wide attention in the field of computer vision owing to the deployment of surveillance system. However, some unfavorable factors restrict the retrieval accuracy of re-ID; minor inter-class difference and orientation variation are two main issues. In this study, we proposed a multi-branch network based on common field of view (CFVMNet) to address these issues. In the proposed method, we extracted and fused the global and local detail features using four branches and the Batch DropBlock (BDB) strategy to accentuate inter-class difference. We also considered some other attributes (i.e., color, type, and model) in the feature extraction process to make the final features more recognizable. For the issue of orientation variation that could lead to large intra-class difference, we learned two different metrics according to whether there is common field of view of two vehicle images, respectively, which can enable the proposed CFVMNet to focus on different regions. Extensive experiments on two public datasets, VeRi-776 and VehicleID, show that the proposed method outperformed the state-of-the-art approaches to vehicle re-ID.
Co-Attentive Lifting for Infrared-Visible Person Re-Identification

Infrared-visible cross-modality person re-identification (IV-ReID) has attracted much attention with the popularity of dual-mode video surveillance systems, where the RGB mode works in the daytime and automatically switches to the infrared mode at night. Despite its significant application value, IV-ReID remains a difficult problem mainly due to two great challenges. First, it is difficult to identify persons in the infrared image, which lacks color and texture clues. Second, there is a significant gap between the infrared and visible modalities where appearances of the same person vary considerably. This paper proposes a novel attention-based approach to handle the two difficulties in a unified framework. 1) We propose an attention lifting mechanism to learn discriminative features in each modality. 2) We propose a co-attentive learning mechanism to bridge the gap between the two modalities. Our method only makes slight modifications of a given backbone network and requires small computation overhead while improving the performance significantly. We conduct extensive experiments to demonstrate the superiority of our proposed method.
P-Companion

If one customer buys a tennis racket, what are the best 3 complementary products to purchase together? 3 tennis ball packs, 3 headbands, 3 overgrips, or 1 of each respectively? Complementary product recommendation (CPR), aiming at providing product suggestions that are often bought together to serve a joint demand, forms a pivotal component of e-commerce service, however, existing methods are far from optimal. Given one product, how to recommend its complementary products of different types is the key problem we tackle in this work. We first conduct an extensive analysis to correct the inaccurate assumptions adopted by existing work to show that co-purchased products are not always complementary and further propose a new strategy to generate clean distant supervision labels for CPR modeling. Moreover, to bridge in the gap from existing work that CPR does not only need relevance modeling but also requires diversity to fulfill the whole purchase demand, we develop a deep learning framework, P-Companion to explicitly model both relevance and diversity. More specifically, given one product with its product type, P-Companion first uses an encoder-decoder network to predict multiple complementary product types, then a transfer metric learning network is developed to project the embedding of query product to each predicted complementary product type subspace and further learn the complementary relationship based on the distant supervision labels within each subspace. The whole framework can be trained from end-to-end and robust to cold-start products attributed to a novel pretrained product embedding module named Product2Vec, based on graph attention networks. Extensive offline experiments show that P-Companion outperforms state-of-the-art baselines by 7.1% increase on the [email protected] score with well-controlled diversity. Production-wise, we deploy P-Companion to provide online recommendations for over 200M products at Amazon and observe significant gains on product sales and profit.
Boosting Visual Question Answering with Context-aware Knowledge Aggregation

Given an image and a natural language question, Visual Question Answering (VQA) aims at answering the textual question correctly. Most VQA approaches in literature targets at finding answers to the questions solely based on analyzing the given images and questions alone. Other works that try to incorporate external knowledge into VQA adopt a query-based search on knowledge graphs to obtain the answer. However, these works suffer from the following problem: the model training process heavily relies on the ground-truth knowledge facts which serve as supervised information --- missing these ground-truth knowledge facts during training will lead to failures in producing the correct answers. To solve the challenging issue, we propose a Knowledge Graph Augmented (KG-Aug) model which conducts context-aware knowledge aggregation on external knowledge graphs, requiring no ground-truth knowledge facts for extra supervision. The proposed KG-Aug model is capable of retrieving context-aware knowledge subgraphs given visual images and textual questions, and learning to aggregate the useful image- and question-dependent knowledge which is then utilized to boost the accuracy in answering visual questions. We carry out extensive experiments to validate the effectiveness of our proposed KG-Aug models against several baseline approaches on various datasets.
Understanding videos at scale: How to extract insights for business research

Video content has become a major component of total internet traffic. Growing bandwidth and computational power conspire with an increasing number of video editing tools, smartphones, and online platforms that have facilitated video production, distribution, and consumption by businesses and consumers alike. This makes video content relevant across business research disciplines. However, analyzing videos can be a cumbersome manual task. Automated techniques are scattered across technical publications and are often not directly accessible to business researchers. This article synthesizes the current state of the art and provides a consolidated tool to efficiently extract 109 video-based variables, requiring no programming knowledge. The variables include structural video characteristics such as colorfulness as well as advanced content-related features such as scene cuts or human face detection. The authors discuss the research potential of video mining, the types of video features of likely interest, and illustrate application using a practical example.
Dynamic Extension Nets for Few-shot Semantic Segmentation

Semantic segmentation requires a large amount of densely annotated data for training and may generalize poorly to novel categories. In real-world applications, we have an urgent need for few-shot semantic segmentation which aims to empower a model to handle unseen object categories with limited data. This task is non-trivial due to several challenges. First, it is difficult to extract the class-relevant information to handle the novel class as only a few samples are available. Second, since the image content can be very complex, the novel class information may be suppressed by the base categories due to limited data. Third, one may easily learn promising base classifiers based on a large amount of training data, but it is non-trivial to exploit the knowledge to train the novel classifiers. More critically, once a novel classifier is built, the output probability space will change. How to maintain the base classifiers and dynamically include the novel classifiers remains an open question. To address the above issues, we propose a Dynamic Extension Network (DENet) in which we dynamically construct and maintain a classifier for the novel class by leveraging the knowledge from the base classes and the information from novel data. More importantly, to overcome the information suppression issue, we design a Guided Attention Module (GAM), which can be plugged into any framework to help learn class-relevant features. Last, rather than directly train the model with limited data, we propose a dynamic extension training algorithm to predict the weights of novel classifiers, which is able to exploit the knowledge of base classifiers by dynamically extending classes during training. The extensive experiments show that our proposed method achieves state-of-the-art performance on the PASCAL-5i and COCO-20i datasets. The source code is available at https://github.com/lizhaoliu-Lec/DENet.
Using the contextual language model BERT for multi-criteria classification of scientific articles

Finding specific scientific articles in a large collection is an important natural language processing challenge in the biomedical domain. Systematic reviews and interactive article search are the type of downstream applications that benefit from addressing this problem. The task often involves screening articles for a combination of selection criteria. While machine learning was previously used for this purpose, it is not known if different criteria should be modeled together or separately in an ensemble model. The performance impact of the modern contextual language models on the task is also not known. We framed the problem as text classification and conducted experiments to compare ensemble architectures, where the selection criteria were mapped to the components of the ensemble. We proposed a novel cascade ensemble analogous to the step-wise screening process employed in developing the gold standard. We compared performance of the ensembles with a single integrated model, which we refer to as the individual task learner (ITL). We used SciBERT, a variant of BERT pre-trained on scientific articles, and conducted experiments using a manually annotated dataset of ~49 K MEDLINE abstracts, known as Clinical Hedges. The cascade ensemble had significantly higher precision (0.663 vs. 0.388 vs. 0.478 vs. 0.320) and F measure (0.753 vs. 0.553 vs. 0.628 vs. 0.477) than ITL and ensembles using Boolean logic and a feed-forward network. However, ITL had significantly higher recall than the other classifiers (0.965 vs. 0.872 vs. 0.917 vs. 0.944). In fixed high recall studies, ITL achieved 0.509 precision @ 0.970 recall and 0.381 precision @ 0.985 recall on a subset that was studied earlier, and 0.295 precision @ 0.985 recall on the full dataset, all of which were improvements over the previous studies. Pre-trained neural contextual language models (e.g. SciBERT) performed well for screening scientific articles. Performance at high fixed recall makes the single integrated model (ITL) more suitable among the architectures considered here, for systematic reviews. However, high F measure of the cascade ensemble makes it a better approach for interactive search applications. The effectiveness of the cascade ensemble architecture suggests broader applicability beyond this task and the dataset, and the approach is analogous to query optimization in Information Retrieval and query optimization in databases.
Transformer-based Label Set Generation for Multi-modal Multi-label Emotion Detection

Multi-modal utterance-level emotion detection has been a hot research topic in both multi-modal analysis and natural language processing communities. Different from traditional single-label multi-modal sentiment analysis, typical multi-modal emotion detection is naturally a multi-label problem where an utterance often contains multiple emotions. Existing studies normally focus on multi-modal fusion only and transform multi-label emotion classification into multiple binary classification problem independently. As a result, existing studies largely ignore two kinds of important dependency information: (1) Modality-to-label dependency, where different emotions can be inferred from different modalities, that is, different modalities contribute differently to each potential emotion. (2) Label-to-label dependency, where some emotions are more likely to coexist than those conflicting emotions. To simultaneously model above two kinds of dependency, we propose a unified approach, namely multi-modal emotion set generation network (MESGN) to generate an emotion set for an utterance. Specifically, we first employ a cross-modal transformer encoder to capture cross-modal interactions among different modalities, and a standard transformer encoder to capture temporal information for each modality-specific sequence given previous interactions. Then, we design a transformer-based discriminative decoding module equipped with modality-to-label attention to handle the modality-to-label dependency. In the meanwhile, we employ a reinforced decoding algorithm with self-critic learning to handle the label-to-label dependency. Finally, we validate the proposed MESGN architecture on a word-level aligned and unaligned multi-modal dataset. Detailed experimentation shows that our proposed MESGN architecture can effectively improve the performance of multi-modal multi-label emotion detection.
Memory-Augmented Relation Network for Few-Shot Learning

Metric-based few-shot learning methods concentrate on learning transferable feature embedding which generalizes well from seen categories to unseen categories under limited supervision. However, most of the methods treat each individual instance separately without considering its relationships with the others in the working context. We investigate a new metric-learning method to explicitly exploit these relationships. In particular, for an instance, we choose the samples that are visually similar from the working context, and perform weighted information propagation to attentively aggregate helpful information from the chosen samples to enhance its representation. We further formulate the distance metric as a learnable relation module which learns to compare for similarity measurement, and equip the working context with memory slots, both contributing to generality. We empirically demonstrate that the proposed method yields significant improvement over its ancestor and achieves competitive or even better performance when compared with other few-shot learning approaches on the two major benchmark datasets, i.e.mini Imagenet andtiered Imagenet.
Blockchain as a service models in the Internet of Things management: Systematic review

Abstract Today, blockchain uses a list of various blocks for storing a distributed flat of invariable information that informs of a set of replicated logical things in the Internet of Things (IoT). In the blockchain, a set of blocks includes various transactions containing a cryptographic hash value and a timestamp. Blockchain‐as‐a‐service (BaaS) as a new service in cloud providers presents an infrastructure for accessing users to execute, manage, and monitor blockchain applications without high secured infrastructure requirements. Recently, BaaS concepts widely have used the management of IoT applications in different layers such as network, data, control, and resource. This paper provides a systematic review of recent research studies in BaaS models. The main goal behind this review is to categorize the applied scenarios, trends, evaluated Quality of Service (QoS) factors, new challenges, and open directions on BaaS models in IoT management. To evaluate the existing research studies in this field, five analytical research questions are proposed to analyze the technical aspects of each study. The analytical results based on existing research questions specify that the BaaS models are applied to network layer to manage IoT environment more than other layers. Also, security and privacy are two important factors to evaluate the existing BaaS models in cloud‐edge IoT environments. Finally, the integration of BaaS models on IoT environments with interconnections of cloud‐edge computing and software defined networks creates great secure opportunities for smart environment applications to monitor, manage, and improve all the atomic services and resources.