Reviewerly/exHarmony huggingface dataset:
Supporting Clustering with Contrastive Learning

Unsupervised clustering aims at discovering the semantic categories of data according to some distance measured in the representation space.However, different categories often overlap with each other in the representation space at the beginning of the learning process, which poses a significant challenge for distance-based clustering in achieving good separation between different categories.To this end, we propose Supporting Clustering with Contrastive Learning (SCCL) -a novel framework to leverage contrastive learning to promote better separation.We assess the performance of SCCL on short text clustering and show that SCCL significantly advances the state-of-the-art results on most benchmark datasets with 3%-11% improvement on Accuracy and 4% -15% improvement on Normalized Mutual Information.Furthermore, our quantitative analysis demonstrates the effectiveness of SCCL in leveraging the strengths of both bottom-up instance discrimination and top-down clustering to achieve better intracluster and inter-cluster distances when evaluated with the ground truth cluster labels 1 .
Blockchain for IoT Applications: Taxonomy, Platforms, Recent Advances, Challenges and Future Research Directions

The Internet of Things (IoT) has become a popular computing technology paradigm. It is increasingly being utilized to facilitate human life processes through a variety of applications, including smart healthcare, smart grids, smart finance, and smart cities. Scalability, interoperability, security, and privacy, as well as trustworthiness, are all issues that IoT applications face. Blockchain solutions have recently been created to help overcome these difficulties. The purpose of this paper is to provide a survey and tutorial on the use of blockchain in IoT systems. The importance of blockchain technology in terms of features and benefits for constituents of IoT applications is discussed. We propose a blockchain taxonomy for IoT applications based on the most significant factors. In addition, we examine the most widely used blockchain platforms for IoT applications. Furthermore, we discuss how blockchain technology can be used to broaden the spectrum of IoT applications. Besides, we discuss the recent advances and solutions offered for IoT environments. Finally, we discuss the challenges and future research directions of the use of blockchain for the IoT.
Evaluation of Adaptive Systems

Adaptive systems are usually interactive systems. As such they stand to benefit considerably from a development lifecycle that ensures user involvement from the early design stages, and embraces evaluation, in both formative and summative forms. Evaluating an adaptive system involves a number of specific problems and pitfalls that need to be addressed by the selection of specific methods, techniques and criteria. This tutorial aims to introduce participants to the peculiarities that arise when evaluating adaptive interactive systems. A layered evaluation framework is used to separate the evaluation process into a number of different aspects which can be applied at different stages throughout the development life-cycle.
TransNFCM: Translation-Based Neural Fashion Compatibility Modeling

Identifying mix-and-match relationships between fashion items is an urgent task in a fashion e-commerce recommender system. It will significantly enhance user experience and satisfaction. However, due to the challenges of inferring the rich yet complicated set of compatibility patterns in a large e-commerce corpus of fashion items, this task is still underexplored. Inspired by the recent advances in multirelational knowledge representation learning and deep neural networks, this paper proposes a novel Translation-based Neural Fashion Compatibility Modeling (TransNFCM) framework, which jointly optimizes fashion item embeddings and category-specific complementary relations in a unified space via an end-to-end learning manner. TransNFCM places items in a unified embedding space where a category-specific relation (category-comp-category) is modeled as a vector translation operating on the embeddings of compatible items from the corresponding categories. By this way, we not only capture the specific notion of compatibility conditioned on a specific pair of complementary categories, but also preserve the global notion of compatibility. We also design a deep fashion item encoder which exploits the complementary characteristic of visual and textual features to represent the fashion products. To the best of our knowledge, this is the first work that uses category-specific complementary relations to model the category-aware compatibility between items in a translation-based embedding space. Extensive experiments demonstrate the effectiveness of TransNFCM over the state-of-the-arts on two real-world datasets.
Margin Learning Embedded Prediction for Video Anomaly Detection with A Few Anomalies

Classical semi-supervised video anomaly detection assumes that only normal data are available in the training set because of the rare and unbounded nature of anomalies. It is obviously, however, these infrequently observed abnormal events can actually help with the detection of identical or similar abnormal events, a line of thinking that motivates us to study open-set supervised anomaly detection with only a few types of abnormal observed events and many normal events available. Under the assumption that normal events can be well predicted, we propose a Margin Learning Embedded Prediction (MLEP) framework. There are three features in MLEP- based open-set supervised video anomaly detection: i) we customize a video prediction framework that favors the prediction of normal events and distorts the prediction of abnormal events; ii) The margin learning framework learns a more compact normal data distribution and enlarges the margin between normal and abnormal events. Since abnormal events are unbounded, our framework consequently helps with the detection of abnormal events, even for anomalies that have never been previously observed. Therefore, our framework is suitable for the open-set supervised anomaly detection setting; iii) our framework can readily handle both frame-level and video-level anomaly annotations. Considering that video-level anomaly detection is more easily annotated in practice and that anomaly detection with a few anomalies is a more practical setting, our work thus pushes the application of anomaly detection towards real scenarios. Extensive experiments validate the effectiveness of our framework for anomaly detection.
A Novel Human Activity Recognition and Prediction in Smart Home Based on Interaction

Smart Homes are generally considered the final solution for living problem, especially for the health care of the elderly and disabled, power saving, etc. Human activity recognition in smart homes is the key to achieving home automation, which enables the smart services to automatically run according to the human mind. Recent research has made a lot of progress in this field; however, most of them can only recognize default activities, which is probably not needed by smart homes services. In addition, low scalability makes such research infeasible to be used outside the laboratory. In this study, we unwrap this issue and propose a novel framework to not only recognize human activity but also predict it. The framework contains three stages: recognition after the activity, recognition in progress, and activity prediction in advance. Furthermore, using passive RFID tags, the hardware cost of our framework is sufficiently low to popularize the framework. In addition, the experimental result demonstrates that our framework can realize good performance in both activity recognition and prediction with high scalability.
An Innovative Big Data Predictive Analytics Framework over Hybrid Big Data Sources with an Application for Disease Analytics

Nowadays, big data are everywhere. Examples of big data include weather data, web-search data, disease reports, as well as epidemic data and statistics. These big data can be easily generated and collected from a wide variety of data sources. A data science framework—such as predictive analytics framework—helps mining data from various big data sources to find useful information and discover knowledge, which can then be transformed into wisdom for appropriate actions. In this paper, we present an innovative big data predictive analytics framework over hybrid big data sources. To demonstrate the effectiveness and practicality of our framework, we conduct several case studies, including one on applying the framework to disease analytics. More specifically, we integrate, incorporate and analyze weather data and web-search data to predict and forecast dengue cases based on a hybrid of three kernels in support vector machine (SVM) ensemble. Results show how our predictive analytics framework benefits health agencies in disease control and prevention.
4DFlowNet: Super-Resolution 4D Flow MRI Using Deep Learning and Computational Fluid Dynamics

4D-flow magnetic resonance imaging (MRI) is an emerging imaging technique where spatiotemporal 3D blood velocity can be captured with full volumetric coverage in a single non-invasive examination. This enables qualitative and quantitative analysis of hemodynamic flow parameters of the heart and great vessels. An increase in the image resolution would provide more accuracy and allow better assessment of the blood flow, especially for patients with abnormal flows. However, this must be balanced with increasing imaging time. The recent success of deep learning in generating super resolution images shows promise for implementation in medical images. We utilized computational fluid dynamics simulations to generate fluid flow simulations and represent them as synthetic 4D flow MRI data. We built our training dataset to mimic actual 4D flow MRI data with its corresponding noise distribution. Our novel 4DFlowNet network was trained on this synthetic 4D flow data and was capable in producing noise-free super resolution 4D flow phase images with upsample factor of 2. We also tested the 4DFlowNet in actual 4D flow MR images of a phantom and normal volunteer data, and demonstrated comparable results with the actual flow rate measurements giving an absolute relative error of 0.6 to 5.8% and 1.1 to 3.8% in the phantom data and normal volunteer data, respectively.
TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task

TACRED (Zhang et al., 2017) is one of the largest, most widely used crowdsourced datasets in Relation Extraction (RE).But, even with recent advances in unsupervised pretraining and knowledge enhanced neural RE, models still show a high error rate.In this paper, we investigate the questions: Have we reached a performance ceiling or is there still room for improvement?And how do crowd annotations, dataset, and models contribute to this error rate?To answer these questions, we first validate the most challenging 5K examples in the development and test sets using trained annotators.We find that label errors account for 8% absolute F1 test error, and that more than 50% of the examples need to be relabeled.On the relabeled test set the average F1 score of a large baseline model set improves from 62.1 to 70.1.After validation, we analyze misclassifications on the challenging instances, categorize them into linguistically motivated error groups, and verify the resulting error hypotheses on three state-of-the-art RE models.We show that two groups of ambiguous relations are responsible for most of the remaining errors and that models may adopt shallow heuristics on the dataset when entities are not masked.
Machine Learning in Healthcare

Recent advancements in Artificial Intelligence (AI) and Machine Learning (ML) technology have brought on substantial strides in predicting and identifying health emergencies, disease populations, and disease state and immune response, amongst a few. Although, skepticism remains regarding the practical application and interpretation of results from ML-based approaches in healthcare settings, the inclusion of these approaches is increasing at a rapid pace. Here we provide a brief overview of machine learning-based approaches and learning algorithms including supervised, unsupervised, and reinforcement learning along with examples. Second, we discuss the application of ML in several healthcare fields, including radiology, genetics, electronic health records, and neuroimaging. We also briefly discuss the risks and challenges of ML application to healthcare such as system privacy and ethical concerns and provide suggestions for future applications.
GeoFarmer: A monitoring and feedback system for agricultural development projects

Farmers can manage their crops and farms better if they can communicate their experiences, both positive and negative, with each other and with experts. Digital agriculture using internet communication technology (ICT) may facilitate the sharing of experiences between farmers themselves and with experts and others interested in agriculture. ICT approaches in agriculture are, however, still out of the reach of many farmers. The reasons are lack of connectivity, missing capacity building and poor usability of ICT applications. We decided to tackle this problem through cost-effective, easy to use ICT approaches, based on infrastructure and services currently available to small-scale producers in developing areas. Working through a participatory design approach, we developed and tested a novel technology. GeoFarmer provides near real-time, two-way data flows that support processes of co-innovation in agricultural development projects. It can be used as a cost-effective ICT-based platform to monitor agricultural production systems with interactive feedback between the users, within pre-defined geographical domains. We tested GeoFarmer in four geographic domains associated with ongoing agricultural development projects in East and West Africa and Latin America. We demonstrate that GeoFarmer is a cost-effective means of providing and sharing opportune indicators of on-farm performance. It is a potentially useful tool that farmers and agricultural practitioners can use to manage their crops and farms better, reduce risk, increase productivity and improve their livelihoods.
DisSent: Learning Sentence Representations from Explicit Discourse Relations

Learning effective representations of sentences is one of the core missions of natural language understanding. Existing models either train on a vast amount of text, or require costly, manually curated sentence relation datasets. We show that with dependency parsing and rule-based rubrics, we can curate a high quality sentence relation task by leveraging explicit discourse relations. We show that our curated dataset provides an excellent signal for learning vector representations of sentence meaning, representing relations that can only be determined when the meanings of two sentences are combined. We demonstrate that the automatically curated corpus allows a bidirectional LSTM sentence encoder to yield high quality sentence embeddings and can serve as a supervised fine-tuning dataset for larger models such as BERT. Our fixed sentence embeddings achieve high performance on a variety of transfer tasks, including SentEval, and we achieve state-of-the-art results on Penn Discourse Treebank’s implicit relation prediction task.
Hyperparameter Optimization in Convolutional Neural Network using Genetic Algorithms

Optimizing hyperparameters in ConvolutionalNeural Network (CNN) is a tedious problem for many researchers and practitioners.To get hyperparameters with better performance, experts are required to configure a set of hyperparameter choices manually.The best results of this manual configuration are thereafter modeled and implemented in CNN.However, different datasets require different model or combination of hyperparameters, which can be cumbersome and tedious.To address this, several works have been proposed such as grid search which is limited to low dimensional space, and tails which use random selection.Also, optimization methods such as evolutionary algorithms and Bayesian have been tested on MNIST datasets, which is less costly and require fewer hyperparameters than CIFAR-10 datasets.In this paper, the authors investigate the hyperparameter search methods on CIFAR-10 datasets.During the investigation with various optimization methods, performances in terms of accuracy are tested and recorded.Although there is no significant difference between propose approach and the state-of-the-art on CIFAR-10 datasets, however, the actual potency lies in the hybridization of genetic algorithms with local search method in optimizing both network structures and network training which is yet to be reported to the best of author knowledge.
Analysing Mathematical Reasoning Abilities of Neural Models

Mathematical reasoning---a core ability within human intelligence---presents some unique challenges as a domain: we do not come to understand and solve mathematical problems primarily on the back of experience and evidence, but on the basis of inferring, learning, and exploiting laws, axioms, and symbol manipulation rules. In this paper, we present a new challenge for the evaluation (and eventually the design) of neural architectures and similar system, developing a task suite of mathematics problems involving sequential questions and answers in a free-form textual input/output format. The structured nature of the mathematics domain, covering arithmetic, algebra, probability and calculus, enables the construction of training and test splits designed to clearly illuminate the capabilities and failure-modes of different architectures, as well as evaluate their ability to compose and relate knowledge and learned processes. Having described the data generation process and its potential future expansions, we conduct a comprehensive analysis of models from two broad classes of the most powerful sequence-to-sequence architectures and find notable differences in their ability to resolve mathematical problems and generalize their knowledge.
Model Based Reinforcement Learning for Atari

Model-free reinforcement learning (RL) can be used to learn effective policies for complex tasks, such as Atari games, even from image observations. However, this typically requires very large amounts of interaction -- substantially more, in fact, than a human would need to learn the same games. How can people learn so quickly? Part of the answer may be that people can learn how the game works and predict which actions will lead to desirable outcomes. In this paper, we explore how video prediction models can similarly enable agents to solve Atari games with fewer interactions than model-free methods. We describe Simulated Policy Learning (SimPLe), a complete model-based deep RL algorithm based on video prediction models and present a comparison of several model architectures, including a novel architecture that yields the best results in our setting. Our experiments evaluate SimPLe on a range of Atari games in low data regime of 100k interactions between the agent and the environment, which corresponds to two hours of real-time play. In most games SimPLe outperforms state-of-the-art model-free algorithms, in some games by over an order of magnitude.
Blockchain Evaluation Approaches: State-of-the-Art and Future Perspective

The present increase of attention toward blockchain-based systems is currently reaching a tipping point with the corporate focus shifting from exploring the technology potential to creating Distributed Ledger Technology (DLT)-based systems. In light of a significant number of already existing blockchain applications driven by the Internet of Things (IoT) evolution, the developers are still facing a lack of tools and instruments for appropriate and efficient performance evaluation and behavior observation of different blockchain architectures. This paper aims at providing a systematic review of current blockchain evaluation approaches and at identifying the corresponding utilization challenges and limitations. First, we outline the main metrics related to the blockchain evaluation. Second, we propose the blockchain modeling and analysis classification based on the critical literature review. Third, we extend the review with publicly accessible industrial tools. Next, we analyze the selected results for each of the proposed classes and outline the corresponding limitations. Finally, we identify current challenges of the blockchain analysis from the system evaluation perspective, as well as provide future perspectives.
Efficient Semantic Video Segmentation with Per-Frame Inference

For semantic segmentation, most existing real-time deep models trained with each frame independently may produce inconsistent results when tested on a video sequence. A few methods take the correlations in the video sequence into account, e.g., by propagating the results to the neighbouring frames using optical flow, or extracting frame representations using multi-frame information, which may lead to inaccurate results or unbalanced latency. In contrast, here we explicitly consider the temporal consistency among frames as extra constraints during training and process each frame independently in the inference phase. Thus no computation overhead is introduced for inference. Compact models are employed for real-time execution. To narrow the performance gap between compact models and large models, new temporal knowledge distillation methods are designed. Weighing among accuracy, temporal smoothness and efficiency, our proposed method outperforms previous keyframe based methods and corresponding baselines which are trained with each frame independently on benchmark datasets including Cityscapes and Camvid. Code is available at: https://git.io/vidseg .
Compressing Large-Scale Transformer-Based Models: A Case Study on BERT

Abstract Pre-trained Transformer-based models have achieved state-of-the-art performance for various Natural Language Processing (NLP) tasks. However, these models often have billions of parameters, and thus are too resource- hungry and computation-intensive to suit low- capability devices or applications with strict latency requirements. One potential remedy for this is model compression, which has attracted considerable research attention. Here, we summarize the research in compressing Transformers, focusing on the especially popular BERT model. In particular, we survey the state of the art in compression for BERT, we clarify the current best practices for compressing large-scale Transformer models, and we provide insights into the workings of various methods. Our categorization and analysis also shed light on promising future research directions for achieving lightweight, accurate, and generic NLP models.