nils-herrmann/clean_scirepeval huggingface dataset:
Hierarchical deep semantic hashing for fast image retrieval
Venue: Multimedia Tools and Applications

Content Based large scale image retrieval has recently attracted considerable attention because of the explosive increase of online images. Inspired by recent advances in convolutional neural networks, we propose a hierarchical deep semantic method for learning similarity function that solves the problems of precision and speed of retrieval in the setting of large scale environments. The distinctive contribution of our work is a novel approach that can utilize previous knowledge of the semantic hierarchy. When semantic information and a related hierarchy structure are available, significant improvements can be attained. Exploiting hierarchical relationships is the most important thing for large scale issues. Binary code can be learned from deep neural network for representing the latent concepts that dominate the semantic labels. Different from other supervised methods that require learning an explicit hashing function to map the binary code features from the images, our method learns Hierarchical Deep Semantic Hashing code (HDSH code) and image representations in an implicit manner, making it suitable for large scale datasets. An additional contribution is a novel hashing scheme (generated at the same time with semantic information) that is able to reduce the computational cost of retrieval. Comprehensive experiments were conducted on Holidays, Oxford5k/105k, Caltech256 retrieval datasets, our HDSH performs competitively even when the convolutional neural network has been pre trained for a surrogate unrelated task. We further demonstrates its efficacy and scalability on a large scale dataset Imagenet with millions of images. With deep hierarchical semantic hashing, we report retrieval times are 0.15ms and 53.92ms on Holidays and Imagenet dataset, respectively.
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Venue: Sensors

Human activity recognition (HAR) tasks have traditionally been solved using engineered features obtained by heuristic processes. Current research suggests that deep convolutional neural networks are suited to automate feature extraction from raw sensor inputs. However, human activities are made of complex sequences of motor movements, and capturing this temporal dynamics is fundamental for successful HAR. Based on the recent success of recurrent neural networks for time series domains, we propose a generic deep framework for activity recognition based on convolutional and LSTM recurrent units, which: (i) is suitable for multimodal wearable sensors; (ii) can perform sensor fusion naturally; (iii) does not require expert knowledge in designing features; and (iv) explicitly models the temporal dynamics of feature activations. We evaluate our framework on two datasets, one of which has been used in a public activity recognition challenge. Our results show that our framework outperforms competing deep non recurrent networks on the challenge dataset by 4% on average; outperforming some of the previous reported results by up to 9% Our results show that the framework can be applied to homogeneous sensor modalities, but can also fuse multimodal sensors to improve performance. We characterise key architectural hyperparameters' influence on performance to provide insights about their optimisation.
Water NOT wanted Coastal Floods and Flooding Protection in Denmark

(28/10/2019) Water NOT wanted Coastal Floods and Flooding Protection in Denmark For living and for leisure we see water as an asset. We are increasingly in favour of living close to water; be it a mountain stream, a tranquil lakeshore, a setting directly on the ocean foreshore or overlooking the ocean scenery. We use our proximity to water for many recreational purposes today. Water is nice! There is a tendency for coastal migration, and a large part of the urbanisation and economic development is taking place in coastal regions throughout the world. This "coastal squeeze" means that we increasingly are exposing ourselves to the forces and hazards of nature in terms of coastal erosion, storm surges, coastal inundation, salination of aquifers etc. Whereas some regions already suffer today, challenges ahead seem immense with projections of sea level rise putting further pressure on our coasts. Although Denmark is normally perceived as a country with a limited vulnerability towards coastal flooding, the country has experienced severe storm surges throughout history, and hitherto safe areas will become increasingly at risk this century as the climate changes. Historically a seafarers' nation, Denmark has always been connected with the sea. From medieval time ports and quays have hosted activities related to maritime trades and harbour workers' quarters. For the past two or three decades these areas have in many towns been rapidly transformed into high end market housing and office facilities. With this transformation, more values have been put at risk and the local acceptance of floods has decreased from a "this is a natural consequence of living by the sea" to an explicit: Water Not Wanted! This paper provides a brief overview of floods and flooding protection issues in Denmark (Ch. 2 Ch. 3) the current legislation (Ch. 4) and discusses challenges in relation to climate change adaptation, risk reduction, and to potential ways of rethinking flooding protection in strategies that also incorporate other uses (Ch. 5)
Increases in Drug and Opioid Involved Overdose Deaths United States, 2010 2015.
Venue: MMWR. Morbidity and mortality weekly report

The U.S. opioid epidemic is continuing, and drug overdose deaths nearly tripled during 1999 2014. Among 47,055 drug overdose deaths that occurred in 2014 in the United States, 28,647 (60.9% involved an opioid (1) Illicit opioids are contributing to the increase in opioid overdose deaths (2,3) In an effort to target prevention strategies to address the rapidly changing epidemic, CDC examined overall drug overdose death rates during 2010 2015 and opioid overdose death rates during 2014 2015 by subcategories (natural/semisynthetic opioids, methadone, heroin, and synthetic opioids other than methadone) Rates were stratified by demographics, region, and by 28 states with high quality reporting on death certificates of specific drugs involved in overdose deaths. During 2015, drug overdoses accounted for 52,404 U.S. deaths, including 33,091 (63.1% that involved an opioid. There has been progress in preventing methadone deaths, and death rates declined by 9.1% However, rates of deaths involving other opioids, specifically heroin and synthetic opioids other than methadone (likely driven primarily by illicitly manufactured fentanyl) (2,3) increased sharply overall and across many states. A multifaceted, collaborative public health and law enforcement approach is urgently needed. Response efforts include implementing the CDC Guideline for Prescribing Opioids for Chronic Pain (4) improving access to and use of prescription drug monitoring programs, enhancing naloxone distribution and other harm reduction approaches, increasing opioid use disorder treatment capacity, improving linkage into treatment, and supporting law enforcement strategies to reduce the illicit opioid supply.
The Contribution of Increased Gamma Band Connectivity to Visual Non Verbal Reasoning in Autistic Children: A MEG Study
Venue: PloS one

Some individuals with autism spectrum (AS) perform better on visual reasoning tasks than would be predicted by their general cognitive performance. In individuals with AS, mechanisms in the brain's visual area that underlie visual processing play a more prominent role in visual reasoning tasks than they do in normal individuals. In addition, increased connectivity with the visual area is thought to be one of the neural bases of autistic visual cognitive abilities. However, the contribution of such brain connectivity to visual cognitive abilities is not well understood, particularly in children. In this study, we investigated how functional connectivity between the visual areas and higher order regions, which is reflected by alpha, beta and gamma band oscillations, contributes to the performance of visual reasoning tasks in typically developing (TD) (n 18) children and AS children (n 18) Brain activity was measured using a custom child sized magneto encephalograph. Imaginary coherence analysis was used as a proxy to estimate the functional connectivity between the occipital and other areas of the brain. Stronger connectivity from the occipital area, as evidenced by higher imaginary coherence in the gamma band, was associated with higher performance in the AS children only. We observed no significant correlation between the alpha or beta bands imaginary coherence and performance in the both groups. Alpha and beta bands reflect top down pathways, while gamma band oscillations reflect a bottom up influence. Therefore, our results suggest that visual reasoning in AS children is at least partially based on an enhanced reliance on visual perception and increased bottom up connectivity from the visual areas.
Neural Machine Translation with External Phrase Memory
Venue: ArXiv

In this paper, we propose phraseNet, a neural machine translator with a phrase memory which stores phrase pairs in symbolic form, mined from corpus or specified by human experts. For any given source sentence, phraseNet scans the phrase memory to determine the candidate phrase pairs and integrates tagging information in the representation of source sentence accordingly. The decoder utilizes a mixture of word generating component and phrase generating component, with a specifically designed strategy to generate a sequence of multiple words all at once. The phraseNet not only approaches one step towards incorporating external knowledge into neural machine translation, but also makes an effort to extend the word by word generation mechanism of recurrent neural network. Our empirical study on Chinese to English translation shows that, with carefully chosen phrase table in memory, phraseNet yields 3.45 BLEU improvement over the generic neural machine translator.
Hybrid computing using a neural network with dynamic external memory
Venue: Nature

Artificial neural networks are remarkably adept at sensory processing, sequence learning and reinforcement learning, but are limited in their ability to represent variables and data structures and to store data over long timescales, owing to the lack of an external memory. Here we introduce a machine learning model called a differentiable neural computer (DNC) which consists of a neural network that can read from and write to an external memory matrix, analogous to the random access memory in a conventional computer. Like a conventional computer, it can use its memory to represent and manipulate complex data structures, but, like a neural network, it can learn to do so from data. When trained with supervised learning, we demonstrate that a DNC can successfully answer synthetic questions designed to emulate reasoning and inference problems in natural language. We show that it can learn tasks such as finding the shortest path between specified points and inferring the missing links in randomly generated graphs, and then generalize these tasks to specific graphs such as transport networks and family trees. When trained with reinforcement learning, a DNC can complete a moving blocks puzzle in which changing goals are specified by sequences of symbols. Taken together, our results demonstrate that DNCs have the capacity to solve complex, structured tasks that are inaccessible to neural networks without external read write memory.
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Venue: ArXiv

Neural Machine Translation (NMT) is an end to end learning approach for automated translation, with the potential to overcome many of the weaknesses of conventional phrase based translation systems. Unfortunately, NMT systems are known to be computationally expensive both in training and in translation inference. Also, most NMT systems have difficulty with rare words. These issues have hindered NMT's use in practical deployments and services, where both accuracy and speed are essential. In this work, we present GNMT, Google's Neural Machine Translation system, which attempts to address many of these issues. Our model consists of a deep LSTM network with 8 encoder and 8 decoder layers using attention and residual connections. To improve parallelism and therefore decrease training time, our attention mechanism connects the bottom layer of the decoder to the top layer of the encoder. To accelerate the final translation speed, we employ low precision arithmetic during inference computations. To improve handling of rare words, we divide words into a limited set of common sub word units "wordpieces" for both input and output. This method provides a good balance between the flexibility of "character" delimited models and the efficiency of "word" delimited models, naturally handles translation of rare words, and ultimately improves the overall accuracy of the system. Our beam search technique employs a length normalization procedure and uses a coverage penalty, which encourages generation of an output sentence that is most likely to cover all the words in the source sentence. On the WMT'14 English to French and English to German benchmarks, GNMT achieves competitive results to state of the art. Using a human side by side evaluation on a set of isolated simple sentences, it reduces translation errors by an average of 60% compared to Google's phrase based production system.
Neural Machine Translation of Rare Words with Subword Units
Venue: ACL

Neural machine translation (NMT) models typically operate with a fixed vocabulary, but translation is an open vocabulary problem. Previous work addresses the translation of out of vocabulary words by backing off to a dictionary. In this paper, we introduce a simpler and more effective approach, making the NMT model capable of open vocabulary translation by encoding rare and unknown words as sequences of subword units. This is based on the intuition that various word classes are translatable via smaller units than words, for instance names (via character copying or transliteration) compounds (via compositional translation) and cognates and loanwords (via phonological and morphological transformations) We discuss the suitability of different word segmentation techniques, including simple character n gram models and a segmentation based on the byte pair encoding compression algorithm, and empirically show that subword models improve over a back off dictionary baseline for the WMT 15 translation tasks English German and English Russian by 1.1 and 1.3 BLEU, respectively.
Therapist youth agreement on alliance change predicts long term outcome in CBT for anxiety disorders.
Venue: Journal of child psychology and psychiatry, and allied disciplines

In individual cognitive behavioral therapy (ICBT) for youth anxiety disorders, it is unclear whether, and from whose perspective, the alliance predicts outcome. We examined whether youth and therapist rated alliance, including level of youth therapist alliance agreement, predicted outcome in a randomized controlled trial.  Youth (N 91, M age 11.4 years (SD 2.1) 49.5% boys, 86.8% Caucasian) diagnosed with separation anxiety disorder, social phobia, or generalized anxiety disorder drawn from the ICBT condition of an effectiveness trial were treated with an ICBT program. Youth and therapist rated alliance ratings, assessed with the Therapeutic Alliance Scale for Children (TASC C/T) were collected following session 3 (early) and 7 (late) Early alliance, change in alliance from early to late, and level of youth therapist agreement on early alliance and alliance change were examined, in relation to outcomes collected at posttreatment and 1 year follow up. Outcome was defined as primary diagnosis loss and reduction in clinicians' severity ratings (CSR; Anxiety Disorders Interview Schedule; ADIS C/P) based on youth and parent report at posttreatment and follow up, and youth treatment satisfaction collected at posttreatment (Client Satisfaction Scale; CSS) RESULTS Early TASC C scores positively predicted treatment satisfaction at posttreatment. Higher levels of agreement on change in TASC C and TASC T scores early to late in treatment predicted diagnosis loss and CSR reduction at follow up.  Only the level of agreement in alliance change predicted follow up outcomes in ICBT for youth anxiety disorders. The findings support further examination of the role that youth therapist alliance discrepancies may play in promoting positive outcomes in ICBT for youth anxiety disorders.
Open science in practice: Learning integrated modeling of coupled surface subsurface flow processes from scratch

Integrated modeling of coupled surface subsurface flow and ensuing role in diverse Earth system processes is of current research interest to characterize nonlinear rainfall runoff response and also to understand land surface energy balances, biogeochemical processes, geomorphological dynamics, etc. A growing number of complex models have been developed for water related research, and many of these are made available to the Earth science community. However, relatively few resources have been made accessible to the potentially large group of Earth science and engineering users. New users have to invest an extraordinary effort to study the models. To provide a stimulating experience focusing on the learning curve of integrated modeling of coupled surface subsurface flow, we describe use cases of an open source model, the Penn State Integrated Hydrologic Model, PIHM. New users were guided through data processing and model application by reproducing a numerical benchmark problem and a real world watershed simulation. Specifically, we document the PIHM application and its computational workflow to enable intuitive understanding of coupled surface subsurface flow processes. In addition, we describe the user experience as important evidence of the significance of reusability. The interaction shows that documentation of data, software, and computational workflow in research papers is a promising method to foster open scientific collaboration and reuse. This study demonstrates how open science practice in research papers would promote the utility of open source software. Addressing such open science practice in publications would promote the utility of journal papers. Further, popularization of such practice will require coordination among research communities, funding agencies, and journals.
Representation Learning of Knowledge Graphs with Entity Descriptions
Venue: AAAI

Representation learning (RL) of knowledge graphs aims to project both entities and relations into a continuous low dimensional space. Most methods concentrate on learning representations with knowledge triples indicating relations between entities. In fact, in most knowledge graphs there are usually concise descriptions for entities, which cannot be well utilized by existing methods. In this paper, we propose a novel RL method for knowledge graphs taking advantages of entity descriptions. More specifically, we explore two encoders, including continuous bag of words and deep convolutional neural models to encode semantics of entity descriptions. We further learn knowledge representations with both triples and descriptions. We evaluate our method on two tasks, including knowledge graph completion and entity classification. Experimental results on real world datasets show that, our method outperforms other baselines on the two tasks, especially under the zero shot setting, which indicates that our method is capable of building representations for novel entities according to their descriptions. The source code of this paper can be obtained from https:/github.com/xrb92/DKRL.
Representation Learning of Knowledge Graphs with Hierarchical Types
Venue: IJCAI

Representation learning of knowledge graphs aims to encode both entities and relations into a continuous low dimensional vector space. Most existing methods only concentrate on learning representations with structured information located in triples, regardless of the rich information located in hierarchical types of entities, which could be collected in most knowledge graphs. In this paper, we propose a novel method named Type embodied Knowledge Representation Learning (TKRL) to take advantages of hierarchical entity types. We suggest that entities should have multiple representations in different types. More specifically, we consider hierarchical types as projection matrices for entities, with two type encoders designed to model hierarchical structures. Meanwhile, type information is also utilized as relation specific type constraints. We evaluate our models on two tasks including knowledge graph completion and triple classification, and further explore the performances on long tail dataset. Experimental results show that our models significantly outperform all baselines on both tasks, especially with long tail distribution. It indicates that our models are capable of capturing hierarchical type information which is significant when constructing representations of knowledge graphs. The source code of this paper can be obtained from https:/github.com/thunlp/TKRL.
VIME: Variational Information Maximizing Exploration
Venue: NIPS

Scalable and effective exploration remains a key challenge in reinforcement learning (RL) While there are methods with optimality guarantees in the setting of discrete state and action spaces, these methods cannot be applied in high dimensional deep RL scenarios. As such, most contemporary RL relies on simple heuristics such as epsilon greedy exploration or adding Gaussian noise to the controls. This paper introduces Variational Information Maximizing Exploration (VIME) an exploration strategy based on maximization of information gain about the agent's belief of environment dynamics. We propose a practical implementation, using variational inference in Bayesian neural networks which efficiently handles continuous state and action spaces. VIME modifies the MDP reward function, and can be applied with several different underlying RL algorithms. We demonstrate that VIME achieves significantly better performance compared to heuristic exploration methods across a variety of continuous control tasks and algorithms, including tasks with very sparse rewards.
Variational Information Maximizing Exploration

Scalable and effective exploration remains a key challenge in reinforcement learning (RL) While there are methods with optimality guarantees in the setting of discrete state and action spaces, these methods cannot be applied in high dimensional deep RL scenarios. As such, most contemporary RL relies on simple heuristics such as greedy exploration or adding Gaussian noise to the controls. This paper introduces Variational Information Maximizing Exploration (VIME) an exploration strategy based on maximization of information gain about the agent's belief of environment dynamics. We propose a practical implementation, using variational inference in Bayesian neural networks which efficiently handles continuous state and action spaces. VIME modifies the MDP reward function, and can be applied with several different underlying RL algorithms. We demonstrate that VIME achieves significantly better performance compared to heuristic exploration methods across a variety of continuous control tasks and algorithms, including tasks with very sparse rewards.