Reviewerly/exHarmony huggingface dataset:
Augmentation of Severe Weather Impact to Far-Infrared Sensor Images to Improve Pedestrian Detection System

Pedestrian detection is an essential task for computer vision and the automotive industry. Complex systems like advanced driver-assistance systems are based on far-infrared data sensors, used to detect pedestrians at nighttime, fog, rain, and direct sun situations. The robust pedestrian detector should work in severe weather conditions. However, only a few datasets include some examples of far-infrared images with distortions caused by atmospheric precipitation and dirt covering sensor optics. This paper proposes the deep learning-based data augmentation technique to enrich far-infrared images collected in good weather conditions by distortions, similar to those caused by bad weather. The six most accurate and fast detectors (TinyV3, TinyL3, You Only Look Once (YOLO)v3, YOLOv4, ResNet50, and ResNext50), performing faster than 15 FPS, were trained on 207,001 annotations and tested on 156,345 annotations, not used for training. The proposed data augmentation technique showed up to a 9.38 mean Average Precision (mAP) increase of pedestrian detection with a maximum of 87.02 mAP (YOLOv4). Proposed in this paper detectors’ Head modifications based on a confidence heat-map gave an additional boost of precision for all six detectors. The most accurate current detector, based on YOLOv4, reached up to 87.20 mAP during our experimental tests.
ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoning

Recent commonsense-reasoning tasks are typically discriminative in nature, where a model answers a multiple-choice question for a certain context.Discriminative tasks are limiting because they fail to adequately evaluate the model's ability to reason and explain predictions with underlying commonsense knowledge.They also allow such models to use reasoning shortcuts and not be "right for the right reasons".In this work, we present EX-PLAGRAPHS, a new generative and structured commonsense-reasoning task (and an associated dataset) of explanation graph generation for stance prediction.Specifically, given a belief and an argument, a model has to predict if the argument supports or counters the belief and also generate a commonsense-augmented graph that serves as non-trivial, complete, and unambiguous explanation for the predicted stance.We collect explanation graphs through a novel Create-Verify-And-Refine graph collection framework that improves the graph quality (up to 90%) via multiple rounds of verification and refinement.A significant 79% of our graphs contain external commonsense nodes with diverse structures and reasoning depths.Next, we propose a multi-level evaluation framework, consisting of automatic metrics and human evaluation, that check for the structural and semantic correctness of the generated graphs and their degree of match with ground-truth graphs.Finally, we present several structured, commonsense-augmented, and text generation models as strong starting points for this explanation graph generation task, and observe that there is a large gap with human performance, thereby encouraging future work for this new challenging task. 1
Fantastic Beasts: Blockchain Based Banking

Blockchain is one of the primary digital technologies utilised in the finance industry with huge future potential. This study conducts a systematic literature review of a final sample of 407 prior literature from an initial set of 1979 records for the sample period of 2013–2020 with regard to blockchain adoption in banking. This review is further supplemented by a machine learning based textual analysis that identifies key themes, trends, divergences and gaps between academic and practitioner led industry literature. Moreover, the study highlights present, future use cases, adoption barriers and misconceptions of blockchains in banking, especially given COVID-19. Furthermore, this study identifies behavioural, social, economic, regulatory and managerial implications of blockchain based banking. In addition, our study identifies the cross-industry potential of blockchains via banking, thus, linking much disconnected prior literature. Finally, we develop a blockchain adoption framework and an adoption life cycle for banking. This study would be of interest to academics, bankers, regulators, investors, auditors and other stakeholders in financial markets.
Blockchain-Enabled Access Management System for Edge Computing

In the post-cloud era, edge computing is a new computing paradigm with data processed at the edge of the network, which can process the data close to the end-user in real time and offload the cloud task intelligently. Meanwhile, the decentralization, tamper-proof and anonymity of blockchain technology can provide a new trusted computing environment for edge computing. However, it does raise considerable concerns of security, privacy, fault-tolerance and so on. For example, identity authentication and access control rely on third parties, heterogeneous devices and different vendors in IoT, leading to security and privacy risks, etc. How to combine the advantages of the two has become the highlight of academic research, especially the issue of secure resource management. Comprehensive security and privacy involve all aspects of platform, data, application and access control. In. this paper, the architecture and behavior of an Access Management System (AMS) in a proof of concept (PoC) prototype are proposed with a Color Petri Net (CPN) model. The two domains of blockchain and edge computing are organically connected by interfaces and interactions. The simulation of operation, activity and role association proves the feasibility and effectiveness of the AMS. The instances of platform business access control, data access control, database services, IOT hub service are run on Advantech WISE-PaaS through User Account and Authentication (UAA). Finally, fine-grained and distributed access control can be realized with the help of a blockchain attribute. Namely, smart contracts are used to register, broadcast, and revoke access authorization, as well as to create specific transactions to define access control policies.
Grounding as a Collaborative Process

Collaborative grounding is a fundamental aspect of human-human dialog which allows people to negotiate meaning.In this paper we argue that it is missing from current deep learning approaches to dialog and interactive systems.Our central point is that making mistakes and being able to recover from them collaboratively is a key ingredient in grounding meaning.We illustrate the pitfalls of being unable to ground collaboratively, discuss what can be learned from the language acquisition and dialog systems literature, and reflect on how to move forward.
Time-Spatial Multiscale Net for Vehicle Counting and Traffic Volume Estimation

Vehicle counting and traffic volume estimation on traffic videos are challenging tasks for traffic monitoring and management. Most previous methods are based on a process of vehicle detection and tracking or a process of time-spatial image (TSI)-based background subtraction, which suffer from occlusion and small size of vehicles. To overcome these difficulties, a time-spatial-based structure is proposed for vehicle counting and traffic volume estimation, without relying on tracking and TSI-based background subtraction. This structure has three main parts. First, a TSI-based density map generation model is proposed for generating density maps for TSIs, which makes it possible to automatically generate TSI training samples. Second, the time-spatial multiscale net (TM-Net) is proposed for estimating density map of TSI; with the stacked multiscale modules and the spatial attention module, the proposed TM-Net can partly overcome the difficulties brought from occlusion, small size, and deformation. Finally, a vehicle counting and traffic volume estimation model is designed for counting and volume estimation on the results of TM-Net. Experiments performed on the public UA-DETRAC data set show that the proposed TM-Net-based vehicle counting method outperforms the tested representative counting methods, and the proposed framework also can estimate traffic volume efficiently.
Unsupervised Deep Keyphrase Generation

Keyphrase generation aims to summarize long documents with a collection of salient phrases. Deep neural models have demonstrated remarkable success in this task, with the capability of predicting keyphrases that are even absent from a document. However, such abstractiveness is acquired at the expense of a substantial amount of annotated data. In this paper, we present a novel method for keyphrase generation, AutoKeyGen, without the supervision of any annotated doc-keyphrase pairs. Motivated by the observation that an absent keyphrase in a document may appear in other places, in whole or in part, we construct a phrase bank by pooling all phrases extracted from a corpus. With this phrase bank, we assign phrase candidates to new documents by a simple partial matching algorithm, and then we rank these candidates by their relevance to the document from both lexical and semantic perspectives. Moreover, we bootstrap a deep generative model using these top-ranked pseudo keyphrases to produce more absent candidates. Extensive experiments demonstrate that AutoKeyGen outperforms all unsupervised baselines and can even beat a strong supervised method in certain cases.
A Novel Deep Learning Based Automated Academic Activities Recognition in Cyber-Physical Systems

Internet of Things (IoT) has rapidly developed in multidisciplinary research topics, particularly in Cyber-Physical infrastructures, such as smart-health care, transportation systems, vehicle management surveillance systems. The smart-video surveillance system has become an essential part of almost all security applications, including academic institutions. University campuses have rich video repositories comprising almost all kinds of academic and non-academic activities. Researchers have introduced many state-of-art activity recognition methods for various application domains with the availability of several activity data sets. Unfortunately, none of these data sets or methods have been developed explicitly for academia and do not cover academic activities. With the advancement of deep learning and IoT, the processing of large-scale video data has become convenient for performing various video analysis tasks. Thus, in this work, an automated deep learning-based academic activities recognition system is presented in smart-cyber infrastructure. We explore a new academic campus domain for research and proposed a novel Convolutional Neural Network (CNN) model for academic activities recognition utilizing a realistic campus dataset. The video database typically contains long, 24-hour video streams recorded by surveillance cameras installed in campus environments. The proposed model’s efficiency is tested through extensive experimentation in terms of accuracy, computation time, and memory requirement. The experimental results reveal that the proposed method attains good results with an accuracy of 98%.
Shape Prior Guided Instance Disparity Estimation for 3D Object Detection

In this paper, we propose a novel system named Disp R-CNN for 3D object detection from stereo images. Many recent works solve this problem by first recovering point clouds with disparity estimation and then apply a 3D detector. The disparity map is computed for the entire image, which is costly and fails to leverage category-specific prior. In contrast, we design an instance disparity estimation network (iDispNet) that predicts disparity only for pixels on objects of interest and learns a category-specific shape prior for more accurate disparity estimation. To address the challenge from scarcity of disparity annotation in training, we propose to use a statistical shape model to generate dense disparity pseudo-ground-truth without the need of LiDAR point clouds, which makes our system more widely applicable. Experiments on the KITTI dataset show that, when LiDAR ground-truth is not used at training time, Disp R-CNN outperforms previous state-of-the-art methods based on stereo input by 20% in terms of average precision for all categories. The code and pseudo-ground-truth data are available at the project page: https://github.com/zju3dv/disprcnn.
Automatic Correction of Real-Word Errors in Spanish Clinical Texts

Real-word errors are characterized by being actual terms in the dictionary. By providing context, real-word errors are detected. Traditional methods to detect and correct such errors are mostly based on counting the frequency of short word sequences in a corpus. Then, the probability of a word being a real-word error is computed. On the other hand, state-of-the-art approaches make use of deep learning models to learn context by extracting semantic features from text. In this work, a deep learning model were implemented for correcting real-word errors in clinical text. Specifically, a Seq2seq Neural Machine Translation Model mapped erroneous sentences to correct them. For that, different types of error were generated in correct sentences by using rules. Different Seq2seq models were trained and evaluated on two corpora: the Wikicorpus and a collection of three clinical datasets. The medicine corpus was much smaller than the Wikicorpus due to privacy issues when dealing with patient information. Moreover, GloVe and Word2Vec pretrained word embeddings were used to study their performance. Despite the medicine corpus being much smaller than the Wikicorpus, Seq2seq models trained on the medicine corpus performed better than those models trained on the Wikicorpus. Nevertheless, a larger amount of clinical text is required to improve the results.
Multi-Channel Generative Framework and Supervised Learning for Anomaly Detection in Surveillance Videos

Recently, most state-of-the-art anomaly detection methods are based on apparent motion and appearance reconstruction networks and use error estimation between generated and real information as detection features. These approaches achieve promising results by only using normal samples for training steps. In this paper, our contributions are two-fold. On the one hand, we propose a flexible multi-channel framework to generate multi-type frame-level features. On the other hand, we study how it is possible to improve the detection performance by supervised learning. The multi-channel framework is based on four Conditional GANs (CGANs) taking various type of appearance and motion information as input and producing prediction information as output. These CGANs provide a better feature space to represent the distinction between normal and abnormal events. Then, the difference between those generative and ground-truth information is encoded by Peak Signal-to-Noise Ratio (PSNR). We propose to classify those features in a classical supervised scenario by building a small training set with some abnormal samples of the original test set of the dataset. The binary Support Vector Machine (SVM) is applied for frame-level anomaly detection. Finally, we use Mask R-CNN as detector to perform object-centric anomaly localization. Our solution is largely evaluated on Avenue, Ped1, Ped2, and ShanghaiTech datasets. Our experiment results demonstrate that PSNR features combined with supervised SVM are better than error maps computed by previous methods. We achieve state-of-the-art performance for frame-level AUC on Ped1 and ShanghaiTech. Especially, for the most challenging Shanghaitech dataset, a supervised training model outperforms up to 9% the state-of-the-art an unsupervised strategy.
PREVIS: Predictive visual analytics of anatomical variability for radiotherapy decision support

Radiotherapy (RT) requires meticulous planning prior to treatment, where the RT plan is optimized with organ delineations on a pre-treatment Computed Tomography (CT) scan of the patient. The conventionally fractionated treatment usually lasts several weeks. Random changes (e.g., rectal and bladder filling in prostate cancer patients) and systematic changes (e.g., weight loss) occur while the patient is being treated. Therefore, the delivered dose distribution may deviate from the planned. Modern technology, in particular image guidance, allows to minimize these deviations, but risks for the patient remain. We present PREVIS: a visual analytics tool for (i) the exploration and prediction of changes in patient anatomy during the upcoming treatment, and (ii) the assessment of treatment strategies, with respect to the anticipated changes. Records of during-treatment changes from a retrospective imaging cohort with complete data are employed in PREVIS, to infer expected anatomical changes of new incoming patients with incomplete data, using a generative model. Abstracted representations of the retrospective cohort partitioning provide insight into an underlying automated clustering, showing main modes of variation for past patients. Interactive similarity representations support an informed selection of matching between new incoming patients and past patients. A Principal Component Analysis (PCA)-based generative model describes the predicted spatial probability distributions of the incoming patient's organs in the upcoming weeks of treatment, based on observations of past patients. The generative model is interactively linked to treatment plan evaluation, supporting the selection of the optimal treatment strategy. We present a usage scenario, demonstrating the applicability of PREVIS in a clinical research setting, and we evaluate our visual analytics tool with eight clinical researchers.
Intelligent Garden Planning and Design Based on Agricultural Internet of Things

As the application of the Internet of Things technology continues to deepen in various fields, it also continues to develop in the agricultural field and the agricultural landscape. This article uses the agricultural Internet of Things to conduct an in-depth analysis and research on the planning and design of smart gardens and combines field research and project practice on the basis of theoretical analysis. Through the combination of agricultural Internet of Things technology and agricultural landscape, the planning and design of smart gardens are studied. The theories used include Internet of Things technology, agricultural industrialization theory, sustainable agricultural development theory, new rural construction theory in agricultural disciplines, ecotourism theory, tourism psychology theory in tourism disciplines, landscape ecology theory, landscape aesthetics theory, and garden planning and design theory. From related theories to planning and design practice, a more systematic study of the smart park has been carried out to provide scientific guidance for the agriculture of the smart park. To provide scientific guidance for the planning and design of agricultural IoT of smart gardens, the theoretical system of planning and design of smart gardens based on the integration of agricultural IoT is proposed and systematically elaborated, including its concept, characteristics, related theories, and guidance methods.
A Linearly Convergent Algorithm for Decentralized Optimization: Sending Less Bits for Free!

Decentralized optimization methods enable on-device training of machine learning models without a central coordinator. In many scenarios communication between devices is energy demanding and time consuming and forms the bottleneck of the entire system. We propose a new randomized first-order method which tackles the communication bottleneck by applying randomized compression operators to the communicated messages. By combining our scheme with a new variance reduction technique that progressively throughout the iterations reduces the adverse effect of the injected quantization noise, we obtain the first scheme that converges linearly on strongly convex decentralized problems while using compressed communication only. We prove that our method can solve the problems without any increase in the number of communications compared to the baseline which does not perform any communication compression while still allowing for a significant compression factor which depends on the conditioning of the problem and the topology of the network. Our key theoretical findings are supported by numerical experiments.
Artificial Intelligence Teaching System and Data Processing Method Based on Big Data

With the rapid development of big data, artificial intelligence teaching systems have gradually been developed extensively. The powerful artificial intelligence teaching systems have become a tool for teachers and students to learn independently in various universities. The characteristic of artificial intelligence teaching system is to get rid of the constraints of traditional teaching time and space and build a brand-new learning environment, which is the mainstream trend of future learning. As the carrier of students’ autonomous learning, the artificial intelligence teaching system provides a wealth of learning resources and learning tools on the one hand, and on the other hand, it gradually accumulates more and more learning behaviors, learning status, and other large amounts of data, which is an in-depth study of online learning and provides valuable and generative dynamic resources. Based on relevant researches on domestic and foreign related learning analysis and common big data analysis methods, combined with actual learning evaluation goals, this paper proposes an artificial intelligence teaching system using big data analysis methods and a modeling process framework for online learning evaluation and uses student data to carry out predictive evaluation modeling to evaluate student learning outcomes. The evaluation results can enable teachers to predict whether students can successfully complete the course of learning after a period of teaching. Through the final evaluation, students’ learning problems can be discovered in time based on the evaluation results, and targeted interventions can be made for students who are at risk. The scientific and objective learning evaluation obtained in this study through data analysis can not only provide teachers with relevant information and provide personalized guidance to students, but also improve the adaptive and personalized service functions of the learning platform of the artificial intelligence teaching system, greatly reducing teachers teaching burden. Artificial intelligence teaching evaluation can help educators understand the problems in teaching, adjust teaching strategies in time, and improve teaching results.
Context-Aware Prosody Correction for Text-Based Speech Editing

Text-based speech editors expedite the process of editing speech recordings by permitting editing via intuitive cut, copy, and paste operations on a speech transcript. A major drawback of current systems, however, is that edited recordings often sound unnatural because of prosody mismatches around edited regions. In our work, we propose a new context-aware method for more natural sounding text-based editing of speech. To do so, we 1) use a series of neural networks to generate salient prosody features that are dependent on the prosody of speech surrounding the edit and amenable to fine-grained user control 2) use the generated features to control a standard pitch-shift and time-stretch method and 3) apply a denoising neural network to remove artifacts induced by the signal manipulation to yield a high-fidelity result. We evaluate our approach using a subjective listening test, provide a detailed comparative analysis, and conclude several interesting insights.