nils-herrmann/clean_scirepeval huggingface dataset:
Improving Machine Reading Comprehension with General Reading Strategies

Reading strategies have been shown to improve comprehension levels, especially for readers lacking adequate prior knowledge. Just as the process of knowledge accumulation is time-consuming for human readers, it is resource-demanding to impart rich general domain knowledge into a language model via pre-training (Radford et al., 2018; Devlin et al., 2018). Inspired by reading strategies identified in cognitive science, and given limited computational resources — just a pre-trained model and a fixed number of training instances — we therefore propose three simple domain-independent strategies aimed to improve non-extractive machine reading comprehension (MRC): (i) BACK AND FORTH READING that considers both the original and reverse order of an input sequence, (ii) HIGHLIGHTING, which adds a trainable embedding to the text embedding of tokens that are relevant to the question and candidate answers, and (iii) SELF-ASSESSMENT that generates practice questions and candidate answers directly from the text in an unsupervised manner. By fine-tuning a pre-trained language model (Radford et al., 2018) with our proposed strategies on the largest existing general domain multiple-choice MRC dataset RACE, we obtain a 5.8% absolute increase in accuracy over the previous best result achieved by the same pre-trained model fine-tuned on RACE without the use of strategies. We further fine-tune the resulting model on a target task, leading to new stateof-the-art results on six representative nonextractive MRC datasets from different domains (i.e., ARC, OpenBookQA, MCTest, MultiRC, SemEval-2018, and ROCStories). These results indicate the effectiveness of the proposed strategies and the versatility ∗ This work was done when the author was an intern at Tencent AI Lab and general applicability of our fine-tuned models that incorporate the strategies.
Cross-domain Semantic Parsing via Paraphrasing

Existing studies on semantic parsing mainly focus on the in-domain setting. We formulate cross-domain semantic parsing as a domain adaptation problem: train a semantic parser on some source domains and then adapt it to the target domain. Due to the diversity of logical forms in different domains, this problem presents unique and intriguing challenges. By converting logical forms into canonical utterances in natural language, we reduce semantic parsing to paraphrasing, and develop an attentive sequence-to-sequence paraphrase model that is general and flexible to adapt to different domains. We discover two problems, small micro variance and large macro variance, of pretrained word embeddings that hinder their direct use in neural networks, and propose standardization techniques as a remedy. On the popular OVERNIGHT dataset, which contains eight domains, we show that both cross-domain training and standardized pre-trained word embedding can bring significant improvement.
A Survey on Deep Learning Based Approaches for Action and Gesture Recognition in Image Sequences

The interest in action and gesture recognition has grown considerably in the last years. In this paper, we present a survey on current deep learning methodologies for action and gesture recognition in image sequences. We introduce a taxonomy that summarizes important aspects of deep learning for approaching both tasks. We review the details of the proposed architectures, fusion strategies, main datasets, and competitions. We summarize and discuss the main works proposed so far with particular interest on how they treat the temporal dimension of data, discussing their main features and identify opportunities and challenges for future research.
An Improved Neural Network Model for Joint POS Tagging and Dependency Parsing

We propose a novel neural network model for joint part-of-speech (POS) tagging and dependency parsing. Our model extends the well-known BIST graph-based dependency parser (Kiperwasser and Goldberg, 2016) by incorporating a BiLSTM-based tagging component to produce automatically predicted POS tags for the parser. On the benchmark English Penn treebank, our model obtains strong UAS and LAS scores at 94.51% and 92.87%, respectively, producing 1.5+% absolute improvements to the BIST graph-based parser, and also obtaining a state-of-the-art POS tagging accuracy at 97.97%. Furthermore, experimental results on parsing 61 “big” Universal Dependencies treebanks from raw texts show that our model outperforms the baseline UDPipe (Straka and Straková, 2017) with 0.8% higher average POS tagging score and 3.6% higher average LAS score. In addition, with our model, we also obtain state-of-the-art downstream task scores for biomedical event extraction and opinion analysis applications. Our code is available together with all pretrained models at: https://github. com/datquocnguyen/jPTDP.
A Simple Sizing Algorithm for Stand-Alone PV / Wind / Battery Hybrid Microgrids

In this paper, we develop a simple algorithm to determine the required number of generating units of wind-turbine generator and photovoltaic array, and the associated storage capacity for stand-alone hybrid microgrid. The algorithm is based on the observation that the state of charge of battery should be periodically invariant. The optimal sizing of hybrid microgrid is given in the sense that the life cycle cost of system is minimized while the given load power demand can be satisfied without load rejection. We also report a case study to show the efficacy of the developed algorithm.
Logistic Regression-HSMM-Based Heart Sound Segmentation

The identification of the exact positions of the first and second heart sounds within a phonocardiogram (PCG), or heart sound segmentation, is an essential step in the automatic analysis of heart sound recordings, allowing for the classification of pathological events. While threshold-based segmentation methods have shown modest success, probabilistic models, such as hidden Markov models, have recently been shown to surpass the capabilities of previous methods. Segmentation performance is further improved when apriori information about the expected duration of the states is incorporated into the model, such as in a hidden semiMarkov model (HSMM). This paper addresses the problem of the accurate segmentation of the first and second heart sound within noisy real-world PCG recordings using an HSMM, extended with the use of logistic regression for emission probability estimation. In addition, we implement a modified Viterbi algorithm for decoding the most likely sequence of states, and evaluated this method on a large dataset of 10 172 s of PCG recorded from 112 patients (including 12 181 first and 11 627 second heart sounds). The proposed method achieved an average F1 score of 95.63 ± 0.85%, while the current state of the art achieved 86.28 ± 1.55% when evaluated on unseen test recordings. The greater discrimination between states afforded using logistic regression as opposed to the previous Gaussian distribution-based emission probability estimation as well as the use of an extended Viterbi algorithm allows this method to significantly outperform the current state-of-the-art method based on a two-sided paired t-test.
Analyzing Gender Stereotyping in Bollywood Movies

The presence of gender stereotypes in many aspects of society is a well-known phenomenon. In this paper, we focus on studying such stereotypes and bias in Hindi movie industry (Bollywood). We analyze movie plots and posters for all movies released since 1970. The gender bias is detected by semantic modeling of plots at inter-sentence and intrasentence level. Different features like occupation, introduction of cast in text, associated actions and descriptions are captured to show the pervasiveness of gender bias and stereotype in movies. We derive a semantic graph and compute centrality of each character and observe similar bias there. We also show that such bias is not applicable for movie posters where females get equal importance even though their character has little or no impact on the movie plot. Furthermore, we explore the movie trailers to estimate on-screen time for males and females and also study the portrayal of emotions by gender in them. The silver lining is that our system was able to identify 30 movies over last 3 years where such stereotypes were broken.
Land Use Classification in Remote Sensing Images by Convolutional Neural Networks

We explore the use of convolutional neural networks for the semantic classification of remote sensing scenes. Two recently proposed architectures, CaffeNet and GoogLeNet, are adopted, with three different learning modalities. Besides conventional training from scratch, we resort to pre-trained networks that are only fine-tuned on the target data, so as to avoid overfitting problems and reduce design time. Experiments on two remote sensing datasets, with markedly different characteristics, testify on the effectiveness and wide applicability of the proposed solution, which guarantees a significant performance improvement over all state-of-the-art references.
Towards a Model of Information Aesthetics in Information Visualization

This paper proposes a model of information aesthetics in the context of information visualization. It addresses the need to acknowledge a recently emerging number of visualization projects that combine information visualization techniques with principles of creative design. The proposed model contributes to a better understanding of information aesthetics as a potentially independent research field within visualization that specifically focuses on the experience of aesthetics, dataset interpretation and interaction. The proposed model is based on analysing existing visualization techniques by their interpretative intent and data mapping inspiration. It reveals information aesthetics as the conceptual link between information visualization and visualization art, and includes the fields of social and ambient visualization. This model is unique in its focus on aesthetics as the artistic influence on the technical implementation and intended purpose of a visualization technique, rather than subjective aesthetic judgments of the visualization outcome. This research provides a framework for understanding aesthetics in visualization, and allows for new design guidelines and reviewing criteria.
Deep Learning Face Representation by Joint Identification-Verification

The key challenge of face recognition is to develop effective feature representations for reducing intra-personal variations while enlarging inter-personal differences. In this paper, we show that it can be well solved with deep learning and using both face identification and verification signals as supervision. The Deep IDentification-verification features (DeepID2) are learned with carefully designed deep convolutional networks. The face identification task increases the inter-personal variations by drawing DeepID2 features extracted from different identities apart, while the face verification task reduces the intra-personal variations by pulling DeepID2 features extracted from the same identity together, both of which are essential to face recognition. The learned DeepID2 features can be well generalized to new identities unseen in the training data. On the challenging LFW dataset [11], 99.15% face verification accuracy is achieved. Compared with the best previous deep learning result [20] on LFW, the error rate has been significantly reduced by 67%.
Understanding the Antecedents of Mobile Game Addiction: The Roles of Perceived Visibility, Perceived Enjoyment and Flow

With the rapid development of mobile games and the roaring growth of market size, mobile game addiction is becoming a public concern. Hence, understanding the reasons behind mobile game addiction is worthwhile. Based on previous studies and two salient features of mobile games (e.g., hedonic and sociality), a research model is developed to examine the antecedents of mobile game addiction. Our proposed model is tested using a survey from 234 mobile game users and the results confirm most of our hypotheses. Specifically, perceived visibility and perceived enjoyment are found to be positively associated with flow which in turn affects addiction. Besides the indirect effect of perceived visibility on addiction via flow, perceived visibility is found to have a direct effect on addiction too. The implications for theory and practice are also discussed.
Caloric restriction and intermittent fasting: Two potential diets for successful brain aging

The vulnerability of the nervous system to advancing age is all too often manifest in neurodegenerative disorders such as Alzheimer's and Parkinson's diseases. In this review article we describe evidence suggesting that two dietary interventions, caloric restriction (CR) and intermittent fasting (IF), can prolong the health-span of the nervous system by impinging upon fundamental metabolic and cellular signaling pathways that regulate life-span. CR and IF affect energy and oxygen radical metabolism, and cellular stress response systems, in ways that protect neurons against genetic and environmental factors to which they would otherwise succumb during aging. There are multiple interactive pathways and molecular mechanisms by which CR and IF benefit neurons including those involving insulin-like signaling, FoxO transcription factors, sirtuins and peroxisome proliferator-activated receptors. These pathways stimulate the production of protein chaperones, neurotrophic factors and antioxidant enzymes, all of which help cells cope with stress and resist disease. A better understanding of the impact of CR and IF on the aging nervous system will likely lead to novel approaches for preventing and treating neurodegenerative disorders.
Intelligent risk management tools for software development

Software tools have been used in software development for a long time now. They are used for, among other things, performance analysis, testing and verification, debugging and building applications. Software tools can be very simple and lightweight, e.g. linkers, or very large and complex, e.g. computer-assisted software engineering (CASE) tools and integrated development environments (IDEs). Some tools support particular phases of the project cycle while others can be used with a speicfic software development model or technology. Some aspects of software development, like risk management, are done throughout the whole project from inception to commissioning. The aim of this paper is to demonstrate the need for an intelligent risk assessment and management tool for both agile or traditional (or their combination) methods in software development. The authors propose a model, whose development is subject of further research, which can be investigated for use in developing intelligent risk management tools
Multi-Cast Attention Networks

Attention is typically used to select informative sub-phrases that are used for prediction. This paper investigates the novel use of attention as a form of feature augmentation, i.e, casted attention. We propose Multi-Cast Attention Networks (MCAN), a new attention mechanism and general model architecture for a potpourri of ranking tasks in the conversational modeling and question answering domains. Our approach performs a series of soft attention operations, each time casting a scalar feature upon the inner word embeddings. The key idea is to provide a real-valued hint (feature) to a subsequent encoder layer and is targeted at improving the representation learning process. There are several advantages to this design, e.g., it allows an arbitrary number of attention mechanisms to be casted, allowing for multiple attention types (e.g., co-attention, intra-attention) and attention variants (e.g., alignment-pooling, max-pooling, mean-pooling) to be executed simultaneously. This not only eliminates the costly need to tune the nature of the co-attention layer, but also provides greater extents of explainability to practitioners. Via extensive experiments on four well-known benchmark datasets, we show that MCAN achieves state-of-the-art performance. On the Ubuntu Dialogue Corpus, MCAN outperforms existing state-of-the-art models by 9%. MCAN also achieves the best performing score to date on the well-studied TrecQA dataset.
WHY DID THE WEST EXTEND THE FRANCHISE ? DEMOCRACY , INEQUALITY , AND GROWTH IN HISTORICAL PERSPECTIVE *

During the nineteenth century most Western societies extended voting rights, a decision that led to unprecedented redistributive programs. We argue that these political reforms can be viewed as strategic decisions by the political elite to prevent widespread social unrest and revolution. Political transition, rather than redistribution under existing political institutions, occurs because current transfers do not ensure future transfers, while the extension of the franchise changes future political equilibria and acts as a commitment to redistribution. Our theory also offers a novel explanation for the Kuznets curve in many Western economies during this period, with the fall in inequality following redistribution due to democratization.
Calibrating Features for Semantic Role Labeling

This paper takes a critical look at the features used in the semantic role tagging literature and show that the information in the input, generally a syntactic parse tree, has yet to be fully exploited. We propose an additional set of features and our experiments show that these features lead to fairly significant improvements in the tasks we performed. We further show that different features are needed for different subtasks. Finally, we show that by using a Maximum Entropy classifier and fewer features, we achieved results comparable with the best previously reported results obtained with SVM models. We believe this is a clear indication that developing features that capture the right kind of information is crucial to advancing the stateof-the-art in semantic analysis.
ADAM-ADMM: A Unified, Systematic Framework of Structured Weight Pruning for DNNs

Weight pruning methods of deep neural networks (DNNs) have been demonstrated to achieve a good model pruning ratio without loss of accuracy, thereby alleviating the significant computation/storage requirements of large-scale DNNs. Structured weight pruning methods have been proposed to overcome the limitation of irregular network structure and demonstrated actual GPU acceleration. However, the pruning ratio (degree of sparsity) and GPU acceleration are limited (to less than 50%) when accuracy needs to be maintained. In this work, we overcome pruning ratio and GPU acceleration limitations by proposing a unified, systematic framework of structured weight pruning for DNNs, named ADAM-ADMM (Adaptive Moment Estimation-Alternating Direction Method of Multipliers). It is a framework that can be used to induce different types of structured sparsity, such as filter-wise, channel-wise, and shape-wise sparsity, as well non-structured sparsity. The proposed framework incorporates stochastic gradient descent with ADMM, and can be understood as a dynamic regularization method in which the regularization target is analytically updated in each iteration. A significant improvement in structured weight pruning ratio is achieved without loss of accuracy, along with fast convergence rate. With a small sparsity degree of 33.3% on the convolutional layers, we achieve 1.64% accuracy enhancement for the AlexNet (CaffeNet) model. This is obtained by mitigation of overfitting. Without loss of accuracy on the AlexNet model, we achieve 2.58× and 3.65× average measured speedup on two GPUs, clearly outperforming the prior work. The average speedups reach 2.77× and 7.5× when allowing a moderate accuracy loss of 2%. In this case the model compression for convolutional layers is 13.2×, corresponding to 10.5× CPU speedup. Our experiments on ResNet model and on other datasets like UCF101 and CIFAR-10 demonstrate the consistently higher performance of our framework. Our models and codes are released at https://github.com/KaiqiZhang/ADAM-ADMM.
Investigation of Wideband Substrate-Integrated Vertically-Polarized Electric Dipole Antenna and Arrays for mm-Wave 5G Mobile Devices

In this work, a mm-Wave vertically-polarized electric dipole array solution for 5G wireless devices is presented. The dipole is fabricated using vias in a standard PCB process to fit at the phone or tablet edges featuring wideband operation with broad half-power beamwidth in the elevation plane (HPBW<sub>ELEV</sub>), high gain and high front-to-back radiation ratio (F/B). For enhanced gain, parasitic-vias are added in front of the dipole as directors. To improve HPBW without sacrificing gain, the directors are implemented as V-shaped bisection parasitic-vias. A via-fence surrounds the dipole structure to suppress back radiation and enhance F/B. The dipole is connected to a parallel-strip line (PS) which is interfaced to the main SIW feed through a novel SIW-to-PS transition. Thorough investigation, optimization, and parametric study are provided for each design parameter of the proposed structure. A single dipole, <inline-formula> <tex-math notation="LaTeX">$2\times 1$ </tex-math></inline-formula>, and <inline-formula> <tex-math notation="LaTeX">$4\times 1$ </tex-math></inline-formula> arrays were designed and fabricated showing close agreement between the simulated and measured results. The single-dipole operates over 7.23-GHz bandwidth with stable radiation performance. The <inline-formula> <tex-math notation="LaTeX">$4\times 1$ </tex-math></inline-formula> array achieves HPBW<sub>ELEV</sub> of 133.1°, F/B of 36.6-dB, cross-polarization less than −39.6-dB and 12.61-dBi gain with 95.8% radiation efficiency. The low cost, compactness, and good performance of the proposed dipole make it a competing candidate for the future 5G mobile devices transceivers.
Bond Risk , Bond Return Volatility , and the Term Structure of Interest Rates

This paper explores time variation in bond risk, as measured by the covariation of bond returns with stock returns and with consumption growth, and in the volatility of bond returns. A robust stylized fact in empirical finance is that the spread between the yield on long-term bonds and short-term bonds forecasts positively future excess returns on bonds at varying horizons, and that the short-term nominal interest rate forecasts positively stock return volatility and exchange rate volatility. This paper presents evidence that movements in both the short-term nominal interest rate and the yield spread are positively related to changes in subsequent realized bond risk and bond return volatility. The yield spread appears to proxy for business conditions, while the short rate appears to proxy for inflation and economic uncertainty. A decomposition of bond betas into a real cash flow risk component, and a discount rate risk component shows that yield spreads have offsetting effects in each component. A widening yield spread is correlated with reduced cash-flow (or inflationary) risk for bonds, but it is also correlated with larger discount rate risk for bonds. The short rate forecasts only the discount rate component of bond beta.  Website http://www.people.hbs.edu/lviceira/. I am grateful to John Campbell, Jakub Jurek, André Perold, participants in the Lisbon International Workshop on the Predictability of Financial Markets, and especially to two anonymous referees and Andréas Heinen for helpful comments and suggestions. I am also very grateful to Johnny Kang for exceptionally able research assistance. I acknowledge the Division of Research at the Harvard Business School for their generous financial support.