Reviewerly/exHarmony huggingface dataset:
Semi-Supervised Sequence-to-Sequence ASR Using Unpaired Speech and Text

Sequence-to-sequence automatic speech recognition (ASR) models require large quantities of data to attain high performance. For this reason, there has been a recent surge in interest for unsupervised and semi-supervised training in such models. This work builds upon recent results showing notable improvements in semi-supervised training using cycle-consistency and related techniques. Such techniques derive training procedures and losses able to leverage unpaired speech and/or text data by combining ASR with Text-to-Speech (TTS) models. In particular, this work proposes a new semi-supervised loss combining an end-to-end differentiable ASR$\rightarrow$TTS loss with TTS$\rightarrow$ASR loss. The method is able to leverage both unpaired speech and text data to outperform recently proposed related techniques in terms of \%WER. We provide extensive results analyzing the impact of data quantity and speech and text modalities and show consistent gains across WSJ and Librispeech corpora. Our code is provided in ESPnet to reproduce the experiments.
Anatomical Attention Guided Deep Networks for ROI Segmentation of Brain MR Images

Brain region-of-interest (ROI) segmentation based on structural magnetic resonance imaging (MRI) scans is an essential step for many computer-aid medical image analysis applications. Due to low intensity contrast around ROI boundary and large inter-subject variance, it has been remaining a challenging task to effectively segment brain ROIs from structural MR images. Even though several deep learning methods for brain MR image segmentation have been developed, most of them do not incorporate shape priors to take advantage of the regularity of brain structures, thus leading to sub-optimal performance. To address this issue, we propose an anatomical attention guided deep learning framework for brain ROI segmentation of structural MR images, containing two subnetworks. The first one is a segmentation subnetwork, used to simultaneously extract discriminative image representation and segment ROIs for each input MR image. The second one is an anatomical attention subnetwork, designed to capture the anatomical structure information of the brain from a set of labeled atlases. To utilize the anatomical attention knowledge learned from atlases, we develop an anatomical gate architecture to fuse feature maps derived from a set of atlas label maps and those from the to-be-segmented image for brain ROI segmentation. In this way, the anatomical prior learned from atlases can be explicitly employed to guide the segmentation process for performance improvement. Within this framework, we develop two anatomical attention guided segmentation models, denoted as anatomical gated fully convolutional network (AG-FCN) and anatomical gated U-Net (AG-UNet), respectively. Experimental results on both ADNI and LONI-LPBA40 datasets suggest that the proposed AG-FCN and AG-UNet methods achieve superior performance in ROI segmentation of brain MR images, compared with several state-of-the-art methods.
Efficient Construction of a Complete Index for Pan-Genomics Read Alignment

Short-read aligners predominantly use the FM-index, which is easily able to index one or a few human genomes. However, it does not scale well to indexing collections of thousands of genomes. Driving this issue are the two chief components of the index: (1) a rank data structure over the Burrows–Wheeler Transform (BWT) of the string that will allow us to find the interval in the string's suffix array (SA), and (2) a sample of the SA that—when used with the rank data structure—allows us to access the SA. The rank data structure can be kept small even for large genomic databases, by run-length compressing the BWT, but until recently there was no means known to keep the SA sample small without greatly slowing down access to the SA. Now that (SODA 2018) has defined an SA sample that takes about the same space as the run-length compressed BWT, we have the design for efficient FM-indexes of genomic databases but are faced with the problem of building them. In 2018, we showed how to build the BWT of large genomic databases efficiently (WABI 2018), but the problem of building the sample efficiently was left open. We compare our approach to state-of-the-art methods for constructing the SA sample, and demonstrate that it is the fastest and most space-efficient method on highly repetitive genomic databases. Lastly, we apply our method for indexing partial and whole human genomes and show that it improves over the FM-index-based Bowtie method with respect to both memory and time and over the hybrid index-based CHIC method with respect to query time and memory required for indexing.
Self-constraining and attention-based hashing network for bit-scalable cross-modal retrieval

Recently deep cross-modal hashing (CMH) have received increased attention in multimedia information retrieval, as it is able to combine the benefit from the low storage cost and search efficiency of hashing, and the strong capabilities of feature abstraction by deep neural networks. CMH can effectively integrates hash representation learning and hash function optimization into an end-to-end framework. However, most of existing deep cross-modal hashing methods use a one-size-fits-all high level representation resulting in the loss of the spatial information of data. Also, previous methods mostly generated fixed length hashing codes. Here, the significance level of different bits are equally weighted thereby restricting their practical flexibility. To address these issues, we propose a self-constraining and attention-based hashing network (SCAHN) for bit-scalable cross-modal hashing. SCAHN integrates the label constraints from early and late-stages as well as their fused features into the hash representation and hash function learning. Moreover, as the fusion of early and late-stages features is based on an attention mechanism, each bit of the hashing codes can be unequally weighted so that the code lengths can be manipulated by ranking the significance of each bit without extra hash-model training. Extensive experiments conducted on four benchmark datasets demonstrate that our proposed SCAHN outperforms the current state-of-the-art CMH methods. Moreover, it is also shown that the generated bit-scalable hashing codes well-preserve the discriminative power with varying code lengths and obtain competitive results comparing to the state-of-the-art.
Comparison of Adaboost with MultiBoosting for Phishing Website Detection

Developments in the Internet introduced new trends. Innovations in Internet bring challenges to our security and expectations of privacy. The anonymous structure of the Internet has such a big attack surface that possess increased risks from cyber-attacks. Attackers introduced novel techniques, such as phishing, to take their vulnerable information such as account IDs, usernames, passwords, etc. Phishing is defined as mimicking a creditable company’s website aiming to take private information of a user. It is a challenge to realize whether a web page is legitimate or phishing, because of its semantics-based attack structure that mostly exploits the Internet users’ vulnerabilities. Several remedies proposed in order to prevent phishing. Although software companies introduced novel anti-phishing techniques that utilize heuristics, blacklists, visual and machine learning methods, only one single magic bullet cannot eliminate these threats entirely. Ensemble machine learning techniques are promising methods employed to distinguish phishing attacks. In this study, an intelligent phishing website detection framework is presented. We employed different machine learning models to classify websites as legitimate or phishing. Several classification techniques were employed in order to implement precise intelligent phishing website detection framework. Classification accuracy, F-measure and area under receiver operating characteristic (ROC) curves (AUC) are utilized to evaluate the performance of the machine learning methods. Experimental results revealed that Adaboost with SVM has outperformed best among the classification methods by achieving the highest accuracy 97.61%.
BasConv: Aggregating Heterogeneous Interactions for Basket Recommendation with Graph Convolutional Neural Network

Within-basket recommendation reduces the exploration time of users, where the user's intention of the basket matters. The intent of a shopping basket can be retrieved from both user-item collaborative filtering signals and multi-item correlations. By defining a basket entity to represent the basket intent, we can model this problem as a basket-item link prediction task in the User-Basket-Item (UBI) graph. Previous work solves the problem by leveraging user-item interactions and item-item interactions simultaneously. However, collectivity and heterogeneity characteristics are hardly investigated before. Collectivity defines the semantics of each node which should be aggregated from both directly and indirectly connected neighbors. Heterogeneity comes from multi-type interactions as well as multi-type nodes in the UBI graph. To this end, we propose a new framework named BasConv, which is based on the graph convolutional neural network. Our BasConv model has three types of aggregators specifically designed for three types of nodes. They collectively learn node embeddings from both neighborhood and high-order context. Additionally, the interactive layers in the aggregators can distinguish different types of interactions. Extensive experiments on two real-world datasets prove the effectiveness of BasConv.
Proactively Identifying Emerging Hacker Threats from the Dark Web

Cybersecurity experts have appraised the total global cost of malicious hacking activities to be $450 billion annually. Cyber Threat Intelligence (CTI) has emerged as a viable approach to combat this societal issue. However, existing processes are criticized as inherently reactive to known threats. To combat these concerns, CTI experts have suggested proactively examining emerging threats in the vast, international online hacker community. In this study, we aim to develop proactive CTI capabilities by exploring online hacker forums to identify emerging threats in terms of popularity and tool functionality. To achieve these goals, we create a novel Diachronic Graph Embedding Framework (D-GEF). D-GEF operates on a Graph-of-Words (GoW) representation of hacker forum text to generate word embeddings in an unsupervised manner. Semantic displacement measures adopted from diachronic linguistics literature identify how terminology evolves. A series of benchmark experiments illustrate D-GEF's ability to generate higher quality than state-of-the-art word embedding models (e.g., word2vec) in tasks pertaining to semantic analogy, clustering, and threat classification. D-GEF's practical utility is illustrated with in-depth case studies on web application and denial of service threats targeting PHP and Windows technologies, respectively. We also discuss the implications of the proposed framework for strategic, operational, and tactical CTI scenarios. All datasets and code are publicly released to facilitate scientific reproducibility and extensions of this work.
Visual Pivoting for (Unsupervised) Entity Alignment

This work studies the use of visual semantic representations to align entities in heterogeneous knowledge graphs (KGs). Images are natural components of many existing KGs. By combining visual knowledge with other auxiliary information, we show that the proposed new approach, EVA, creates a holistic entity representation that provides strong signals for cross-graph entity alignment. Besides, previous entity alignment methods require human labelled seed alignment, restricting availability. EVA provides a completely unsupervised solution by leveraging the visual similarity of entities to create an initial seed dictionary (visual pivots). Experiments on benchmark data sets DBP15k and DWY15k show that EVA offers state-of-the-art performance on both monolingual and cross-lingual entity alignment tasks. Furthermore, we discover that images are particularly useful to align long-tail KG entities, which inherently lack the structural contexts necessary for capturing the correspondences. Code release: https://github.com/cambridgeltl/eva; project page: http://cogcomp.org/page/publication view/927.
Kidney and Renal Tumor Segmentation Using a Hybrid V-Net-Based Model

Kidney tumors represent a type of cancer that people of advanced age are more likely to develop. For this reason, it is important to exercise caution and provide diagnostic tests in the later stages of life. Medical imaging and deep learning methods are becoming increasingly attractive in this sense. Developing deep learning models to help physicians identify tumors with successful segmentation is of great importance. However, not many successful systems exist for soft tissue organs, such as the kidneys and the prostate, of which segmentation is relatively difficult. In such cases where segmentation is difficult, V-Net-based models are mostly used. This paper proposes a new hybrid model using the superior features of existing V-Net models. The model represents a more successful system with improvements in the encoder and decoder phases not previously applied. We believe that this new hybrid V-Net model could help the majority of physicians, particularly those focused on kidney and kidney tumor segmentation. The proposed model showed better performance in segmentation than existing imaging models and can be easily integrated into all systems due to its flexible structure and applicability. The hybrid V-Net model exhibited average Dice coefficients of 97.7% and 86.5% for kidney and tumor segmentation, respectively, and, therefore, could be used as a reliable method for soft tissue organ segmentation.
Time series forecasting based on echo state network and empirical wavelet transformation

Echo state network (ESN) is a reservoir computing structure consisting randomly generated hidden layer which enables a rapid learning and extrapolation process. On the other hand, the determination of inputs is still an outstanding question similar to other neural networks. Instead of utilizing the original historical data, wavelet transformations can be used to extract and reflect features with predictability (periodicity) and eliminate random oscillations (i.e. noise) which would distort the underlying predictive structure. In this regard, we propose a two-stage predictive algorithm in which the empirical wavelet transformation (EWT) has been implemented to transform the data, and ESN is utilized to execute the overall predictive process. The proposed method has been validated by twelve public datasets with different mean-volatility features. Out of sample forecasts have been compared to the baseline persistence model and conventional benchmarks. The empirical study demonstrates the superiority of the EWT–ESN model compared with other models.
Cross-lingual Contextualized Topic Models with Zero-shot Learning

Many data sets (e.g., reviews, forums, news, etc.) exist parallelly in multiple languages.They all cover the same content, but the linguistic differences make it impossible to use traditional, bag-of-word-based topic models.Models have to be either single-language or suffer from a huge, but extremely sparse vocabulary.Both issues can be addressed by transfer learning.In this paper, we introduce a zero-shot cross-lingual topic model.Our model learns topics on one language (here, English), and predicts them for unseen documents in different languages (here, Italian, French, German, and Portuguese).We evaluate the quality of the topic predictions for the same document in different languages.Our results show that the transferred topics are coherent and stable across languages, which suggests exciting future research directions.
S<scp>oloist</scp>: BuildingTask Bots at Scale with Transfer Learning and Machine Teaching

Abstract We present a new method, Soloist,1 that uses transfer learning and machine teaching to build task bots at scale. We parameterize classical modular task-oriented dialog systems using a Transformer-based auto-regressive language model, which subsumes different dialog modules into a single neural model. We pre-train, on heterogeneous dialog corpora, a task-grounded response generation model, which can generate dialog responses grounded in user goals and real-world knowledge for task completion. The pre-trained model can be efficiently adapted to accomplish new tasks with a handful of task-specific dialogs via machine teaching, where training samples are generated by human teachers interacting with the system. Experiments show that (i)Soloist creates new state-of-the-art on well-studied task-oriented dialog benchmarks, including CamRest676 and MultiWOZ; (ii) in the few-shot fine-tuning settings, Soloist significantly outperforms existing methods; and (iii) the use of machine teaching substantially reduces the labeling cost of fine-tuning. The pre-trained models and codes are available at https://aka.ms/soloist.
Topic-Driven and Knowledge-Aware Transformer for Dialogue Emotion Detection

Emotion detection in dialogues is challenging as it often requires the identification of thematic topics underlying a conversation, the relevant commonsense knowledge, and the intricate transition patterns between the affective states.In this paper, we propose a Topic-Driven Knowledge-Aware Transformer to handle the challenges above.We firstly design a topic-augmented language model (LM) with an additional layer specialized for topic detection.The topic-augmented LM is then combined with commonsense statements derived from a knowledge base based on the dialogue contextual information.Finally, a transformerbased encoder-decoder architecture fuses the topical and commonsense information, and performs the emotion label sequence prediction.The model has been experimented on four datasets in dialogue emotion detection, demonstrating its superiority empirically over the existing state-of-the-art approaches.Quantitative and qualitative results show that the model can discover topics which help in distinguishing emotion categories.
Sentiment Analysis of Nepali COVID19 Tweets Using NB, SVM AND LSTM

The government's months-long total lockdown in response to the COVID19 outbreak has resulted in a lack of physical connection with others. This resulted in a massive increase in social media communication. Twitter has become one of the most popular places for people to communicate their thoughts and opinions. As a result, massive amounts of data are created every day. These data can assist businesses in making better judgments. In the case of Nepal, there has been relatively little investigation into the text's analysis. Because few researchers are working in the field, development is slow. In this study, Four language-based models for sentiment analysis of Nepali covid19 tweets are designed and evaluated. Because the number of individuals using social media is expected to skyrocket in the next few days, companies will benefit from an AI-based sentiment analysis system. It will greatly assist firms in adapting to the changing climate.
Learning with Privileged Information via Adversarial Discriminative Modality Distillation

Heterogeneous data modalities can provide complementary cues for several tasks, usually leading to more robust algorithms and better performance. However, while training data can be accurately collected to include a variety of sensory modalities, it is often the case that not all of them are available in real life (testing) scenarios, where a model has to be deployed. This raises the challenge of how to extract information from multimodal data in the training stage, in a form that can be exploited at test time, considering limitations such as noisy or missing modalities. This paper presents a new approach in this direction for RGB-D vision tasks, developed within the adversarial learning and privileged information frameworks. We consider the practical case of learning representations from depth and RGB videos, while relying only on RGB data at test time. We propose a new approach to train a hallucination network that learns to distill depth information via adversarial learning, resulting in a clean approach without several losses to balance or hyperparameters. We report state-of-the-art results for object classification on the NYUD dataset, and video action recognition on the largest multimodal dataset available for this task, the NTU RGB+D, as well as on the Northwestern-UCLA.
2019 Data Fusion Contest [Technical Committees]

Presents information on the 2019 Data Fusion Contest.
Bitcoin’s energy consumption: Is it the Achilles heel to miner’s revenue?

In this paper, we add a new dimension to the cryptocurrency literature by examining the relationship between Bitcoin’s energy consumption and miner’s revenue. By resorting to quantile and Markov regime switching regression, we report a negative association between these variables. Further, the negative impact is strongly significant when the miner’s revenues are low and volatile. Thus, the higher energy consumption in the wake of escalating global energy costs amid bearish market sentiments impedes the miners to break-even. Hence, it would not be viable to sustain the business unless cheap energy sources and efficient mining hardware are relied upon.
A hybrid approach with optimization-based and metric-based meta-learner for few-shot learning

Few-shot learning aims to learn classifiers for new classes with only a few training examples per class. Most existing few-shot learning approaches belong to either metric-based meta-learning or optimization-based meta-learning category, both of which have achieved successes in the simplified “k-shot N-way” image classification settings. Specifically, the optimization-based approaches train a meta-learner to predict the parameters of the task-specific classifiers. The task-specific classifiers are required to be homogeneous-structured to ease the parameter prediction, so the meta-learning approaches could only handle few-shot learning problems where the tasks share a uniform number of classes. The metric-based approaches learn one task-invariant metric for all the tasks. Even though the metric-learning approaches allow different numbers of classes, they require the tasks all coming from a similar domain such that there exists a uniform metric that could work across tasks. In this work, we propose a hybrid meta-learning model called Meta-Metric-Learner which combines the merits of both optimization- and metric-based approaches. Our meta-metric-learning approach consists of two components, a task-specific metric-based learner as a base model, and a meta-learner that learns and specifies the base model. Thus our model is able to handle flexible numbers of classes as well as generate more generalized metrics for classification across tasks. We test our approach in the standard “k-shot N-way” few-shot learning setting following previous works and a new realistic few-shot setting with flexible class numbers in both single-source form and multi-source form. Experiments show that our approach attains superior performance in all settings.