Miracle-dz/UniFAR-FTUs huggingface dataset:
Surviving the Fintech Disruption

This paper studies how demand for labor reacts to financial technology (fintech) shocks based on comprehensive databases of fintech patents and firm job postings in the U.S. during the past decade. We first develop a measure of fintech exposure at the occupation level by intersecting the textual information in job task descriptions and fintech patents. We then document a significant decline of job postings in the most exposed occupations, and an increase in the industry as well as the geographical concentration of these occupations. Firms resort to an upskilling strategy in face of the fintech disruption, requiring "combo" (finance and software) skills, higher education attainments, and longer work experiences in the hiring of fintech-exposed jobs. Financial firms and those with high innovation outputs are able to offset the disruptive effect from the fintech shock. Among innovating firms, however, only inventors (but not acquisition-driven innovators) experience growth in hiring, sales, investment, and enjoy better returns on assets.
CIBench: A Dataset and Collection of Techniques for Build and Test Selection and Prioritization in Continuous Integration

Continuous integration (CI) is a widely used practice in modern software engineering. Unfortunately, it is also an expensive practice — Google and Mozilla estimate their CI systems in millions of dollars. There are a number of techniques and tools designed to or having the potential to save the cost of CI or expand its benefit - reducing time to feedback. However, their benefits in some dimensions may also result in drawbacks in others. They may also be beneficial in other scenarios where they are not designed to help. Therefore, we built CIBench, a dataset and collection of techniques for build and test selection and prioritization in Continuous Integration. CIBench is based on a popular existing dataset for CI — TravisTorrent [2] and extends it in multiple ways including mining additional Travis logs, Github commits, and building dependency graphs for studied projects. This dataset allows us to replicate and evaluate existing techniques to improve CI under the same settings, to better understand the impact of applying different strategies in a more comprehensive way.
Understanding the Educational Attainment Polygenic Score and Its Interactions with SES in Determining Health in Young Adulthood∗

We investigate an Educational Attainment Polygenic Score (EA PGS), an index that predicts years of formal education based on genetic data. In our analysis of the National Longitudinal Study of Adolescent to Adult Health we find that the EA PGS affects a number of health-related outcomes. Moreover, the EA PGS interacts with parental socioeconomic status (SES) in childhood: for a number of health outcomes we observe that the effect of the EA PGS is more beneficial for high-SES subjects. We decompose the total effects of the EA PGS into the indirect effect (through education) and the direct effect. We also decompose both the direct and the total effect with respect to potential mechanisms. The mechanisms that partially explain the effects of EA PGS include early skills, early health, education support in the family, and education. As a result of our discovery of a strong direct effect we cast our doubts on the validity of the EA PGS used as an instrumental variable for education affecting health, a case of an increasingly utilized technique called Mendelian Randomization. Finally, after controlling for the EA PGS, genetic health endowments, and unobserved heterogeneity in addition to more traditional controls, we still find that education is associated with better health outcomes, which adds evidence to the ongoing debate about the causal link between education and health.
Meta Learning with Data Augmentation for Robust Out-of-domain Question Answering

Natural language understanding problems has gain much popularity over the years and current models often has poor generalizability on the out-of-domain tasks. This robust question answering (QA) project aims to remedy this situation by using Reptile, a variant of meta learning algorithms. In this project, the primary goal is to implement Reptile algorithm for question answering tasks to achieve a better performance than the baseline model on the out-of-domain datasets. After the Reptile implementation is validated, the secondary goal of this project is to explore how various hyper parameters affect the final performance. After we believe that the Reptile is optimally tuned, we worked on the data provided for this project. First, we merged in-domain validation dataset to the training data, then we added data augmentation to further tap into the potential of the out-of-domain training data. Training results of Reptile outperforms vanilla BERT model and Reptile with data augmentation increases the score even further. The best F1 score is 59.985 and best EM score is 42.225. If we compare the performance on out-of-domain validation dataset, scores are more than 12% and 22% higher than the baseline score respectively.
Robust Question Answering Through Data Augmentation and TAPT

In this project, we aim to improve on the given baseline model, which is a Distil-BERT pretained transformer, as much as possible in order to make it more robust to out-of-domain data for the task of QA. In order to do this, we experimented with a variety of extensions to the baseline, among which are Task-Adaptive Pretraining [1] and data augmentation. We found that data augmentation was able to improve the results of the baseline the best out of our various attempts. Our best model performed better than the baseline by 0.287 points for the Fl score and 0.941 points for the EM score on the test set. The code for this project can be found on this Github repository: https://github.com/duniahakim/RobustQA. It is a private repository for Honor Code reasons but let me me know if you would like me to invite you as a collaborator to view it.
the effect of massage therapy on fatigue of patients with multiple sclerosis

1 Faculty Member of Nursing and Midwifery, Bam University of Medical Sciences, Bam, Iran. 2Physiology Research Center of Kerman University of Medical Sciences, Kerman, Iran, 3Assistant Professor, Faculty of Medicine, Bam University of Medical Sciences, Bam, Iran. 4 MSC Crtical Care Nursing, Kerman, Iran. 5Bachelor of Nursing, ShahidBahonar Hospital, Kerman University of Medical Sciences, Kerman, Iran.6 Faculty Member of Nursing Department, Faculty of Nursing, Midwifery and Health, Islamic Azad University, Kerman, Iran.
Towards a Robust Question Answering System through Domain-adaptive Pretraining and Data Augmentation

Large language models (LMs) have shown great success over a bunch of tasks in the past few years. These large LMs are trained on enormous corpus, and it now becomes a question whether they are robust to domain shift. We find in this paper that the domain of question answering (QA) problems has significant impact on the performance of these fine-tuned LMs and these fine-tuned QA models are still sensitive to domain shift during test time. This potentially causes problems in many real-word applications where broad or evolving domains are involved. In this paper, we offer two potential solutions towards building a more robust QA system with domain shift. First, we propose to continue pretraining the LMs on the objective domains. We find that domain-adaptive pretraining helps improve out-of-domain test performance. In some cases, we might have additional small amount of training data on the test domain. We propose to use data augmentation tricks to maximally utilize these data for domain adaptation purpose. We find that data augmentation tricks, including synonym replacement, random insertion and random deletion, can further improve the performance on out-of-domain test samples. Our work shows that the improvements in performance from domain-adaptive pretraining and data augmentation are additive. With both methods applied, our model achieves a test performance of 60.731 in F1 score and 42.248 in EM score. The experiments and methods discussed in this paper will contribute to a deeper understanding of LMs and efforts towards building a more robust QA system.
Using Semantics of Textbook Highlights to Predict Student Comprehension and Knowledge Retention

As students read textbooks, they often highlight the material they deem to be most important. We analyze students’ highlights to predict their subsequent performance on quiz questions. Past research in this area has encoded highlights in terms of where the highlights appear in the stream of text—a positional representation. In this work, we construct a semantic representation based on a state-of-the-art deep-learning sentence embedding technique (SBERT) that captures the content-based similarity between quiz questions and highlighted (as well as non-high-lighted) sentences in the text. We construct regression models that include latent variables for student skill level and question diﬃculty and augment the models with highlighting features. We ﬁnd that highlighting features reliably boost model performance. We conduct experiments that validate models on held-out questions, students, and student-questions and ﬁnd strong generalization for the latter two but not for held-out questions. Surprisingly, highlighting features improve models for questions at all levels of the Bloom taxonomy, from straightforward recall questions to inferential synthesis/evaluation/creation questions.
Is There Income Inequality Convergence Between Regions in Turkey? Panel Data Analysis

Son yillarda kuresellesme ve teknolojik degisim ile birlikte bircok ekonomide gelir esitsizliginin arttigi gorulmektedir. Bu yuzden gelir esitsizligi kavrami arastirmacilar tarafindan dikkat ceken konulardan biri olmaktadir. Bu calismada Turkiye icin bolgeler arasi gelir dagilimi esitsizliginin stokastik yakinsama hipotezinin gecerliligi sinanmak istenmektedir. Bu amacla gelir esitsizliginin duraganlik sinamasi panel birim kok testi ile gerceklestirilmistir. Gini katsayisi olarak ifade edilen gelir esitsizligi serileri 2006-2019 donemini kapsamaktadir ve bu katsayiya Turkiye Istatistik Kurumu resmi veri tabanindan erisilmistir. Istatistiki bolge birimleri siniflamasina gore 12 Bolge (Duzey 1) icin yapilan panel birim kok analizi neticesinde Gini katsayisinin duragan olmadigi sonucuna varilmistir. Yani Turkiye’de bolgeler arasi yakinsama hipotezi gecerli degildir ve gelir dagiliminda dengesizlik mevcuttur. Dolayisiyla gelir dagiliminda adaleti saglamak icin politika yapicilara cok is dusmektedir.
QANet without Backtranslation on SQUAD 2.0

This paper investigates two different approaches to the question answering problem on the SQuAD 2.0 dataset [1]. We explore a baseline model based on the BiDaF architecture [2], and improve its performance through the implementation of character embeddings and hyperparameter tuning. Further, we implement variations on the convolution and self-attention based QANéet architecture [3], and explore an alternative technique for data augmentation. Finally, we form an ensemble model based on our different experiments which achieves an F1 score of 70.340 and an EM score of 67.354 on the test set.
Extending QANéet with Transformer-XL

This default project tackles the machine reading comprehension (RC) problem on the SQUAD 2.0 dataset [1]. It involves inputting a context paragraph and a question into a model and outputting the span of the answer from the context paragraph. This project aims to extend the QANet [2], so that it can effectively perform RC on SQuAD 2.0. The segment-level recurrence with state reuse from Transformer-XL [3] is integrated into QANet to improve its ability of tackling long context paragraph (referred to as QANet-XL). In addition, character embeddings and a fusion layer after context-query attention are used to extend the BiDAF [4] baseline. Experiments show that QANet-XL underperforms the vanilla QANeéet and outperforms the extended BiDAF. The segment-level recurrence mechanism from Transformer-XL is proven not a proper improvement for QANet on the SQuAD 2.0 dataset, since Transformer-XL is designed for extra-long continuous text. For the dev set, The extended BiDAF achieved EM/F 1 = 62.16/65.98, the vanilla QANet achieved EM/F1 = 66.81/70.38, and the QANet-XL achieved EM/F1 = 63.12/66.67. A majority voting ensemble model based on previous mentioned models achieved EM/F1 = 66.85/69.97 on the test set.
CS224N Default Final Project Report: Building a QA System Using BiDAF and Subword Modeling Techniques

Recently, the task of question answering (QA) has gained significant popularity in the field of natural language processing. For this project, we explore deep learning architectures, in particular the Bi-Directional Attention Flow (BiDAF) network, to construct a better-performing model than the baseline we were provided with. Our baseline model achieved 57.54 EM and 60.90 F1 in the dev set. Based on this, we experimented with concatenating character embeddings with word embeddings and other forms of subword modeling. We implemented a subword vocabulary with Byte-Pair Encoding and made a corresponding embedding layer, but found that it had suboptimal performence, likely due to its potential confusion with out-of-vocabulary words in the dev and test datasets. Our final system and best-performing model is the BiDAF network with the character embedding layer, where character and word embeddings are concatenated in equal part (50/50). Our best results achieved 60.595 EM and 63.587 F1 on the dev set and 59.222 EM and 62.662 F1 on the test set.
Regulatory Capital Modelling for Credit Risk

The Basel II internal ratings-based (IRB) approach to capital adequacy for credit risk plays an important role in protecting the Australian banking sector against insolvency. We outline the mathematical foundations of regulatory capital for credit risk, and extend the model specification of the IRB approach to a more general setting than the usual Gaussian case. It rests on the proposition that quantiles of the distribution of conditional expectation of portfolio percentage loss may be substituted for quantiles of the portfolio loss distribution. We present a more economical proof of this proposition under weaker assumptions. Then, constructing a portfolio that is representative of credit exposures of the Australian banking sector, we measure the rate of convergence, in terms of number of obligors, of empirical loss distributions to the asymptotic (infinitely fine-grained) portfolio loss distribution. Moreover, we evaluate the sensitivity of credit risk capital to dependence structure as modelled by asset correlations and elliptical copulas. Access to internal bank data collected by the prudential regulator distinguishes our research from other empirical studies on the IRB approach.
Loop equations , matrix models , and N = 1 supersymmetric gauge theories

We derive the Konishi anomaly equations for N = 1 supersymmetric gauge theories based on the classical gauge groups with matter in two-index tensor and fundamental representations, thus extending the existing results for U(N). A general formula is obtained which expresses solutions to the Konishi anomaly equation in terms of solutions to the loop equations of the corresponding matrix model. This provides an alternative to the diagrammatic proof that the perturbative part of the glueball superpotential Weff for these matter representations can be computed from matrix model integrals, and further shows that the two approaches always give the same result. The anomaly approach is found to be computationally more efficient in the cases we studied. Also, we show in the anomaly approach how theories with a traceless two-index tensor can be solved using an associated theory with a traceful tensor and appropriately chosen coupling constants.
Investigating Techniques for Improving NMT Systems for Low Resource Languages

Neural Machine Translation (NMT) has become the standard for Machine Translation tasks, however, they encounter many technical challenges when training in low resource language pairs. In this paper, we investigate how different subword and word representations, as well as different data augmentation techniques can improve NMT performance on low resource languages. For our baseline, we train an encoder-decoder based seq2seq NMT model on a scarce Nepali-English dataset. Then, we compare different subword and word representations, such as Byte Pair Encoding (BPE) and a reduced vocab set. Finally, we augment our training data with backtranslation of monolingual data, transfer learning from Hindi, and noisy data. In addition, we propose a new variant of backtranslation for low-resource NMT that exceeds performance of traditional backtranslation methods. We find that BPE was the best performing subword representation. For data augmentation, we find that transfer learning and noisy data gives reliable improvements, yet back translation requires careful management of noise levels. By utilizing our novel variant of backtranslation alongside BPE and auxiliary data methods in combined models, we are able to increase in-domain performance by +4.55 BLEU and out-of-domain performance by +3.93 BLEU compared to the baseline.
Hierarchical, Feature-Based Text Generation

We introduce the creation and use of full-text, distributed feature maps as the basis for the hierarchical generation of long-form text. The problem of story generation, a challenge that consists of generating narratively-coherent passages of text about a particular topic, can be described as one of the most difficult challenges currently posed in text generation, as stories require long-range dependencies, creativity, and a high-level plot. Previous efforts note that story generation often fails to meet one or more of these requirements; generated stories are frequently repetitive, and typically lack any kind of a broader arc. We find that the use of automatically-generated "emotion maps" as a basis for hierarchical generation achieve perplexity scores comparable to previous efforts, despite using a numerical input rather than a textual one. Additionally, we introduce a new story generation dataset, consisting of 100,000 one thousand word stories, each paired with a series of tags which contain genre, character, and other feature information. We demonstrate that use of fully quantifiable feature maps as a conditional basis for generation achieves results comparable to the state of the art on multiple datasets. We also introduce a method for quantifying feature map/story relationship, and use this metric to show that the feature maps have a limited, but extant relationship to the generated text. Future use of quantitative analysis in hierarchical generation will aid researchers in effectively constructing and using first-step prompts for story generation.
1 Network mechanisms of intentional learning

The ability to learn new tasks rapidly is a prominent characteristic of human behaviour. This ability relies on flexible cognitive systems that adapt in order to encode temporary programs for processing non-automated tasks. Previous functional imaging studies have revealed distinct roles for the lateral frontal cortices (LFCs) and the ventral striatum in intentional learning processes. However, the human LFCs are complex; they house multiple distinct sub-regions, each of which co-activates with a different functional network. It remains unclear how these LFC networks differ in their functions and how they coordinate with each other, and the ventral striatum, to support intentional learning. Here, we apply a suite of fMRI connectivity methods to determine how LFC networks activate and interact at different stages of two novel tasks, in which arbitrary stimulus-response rules are learnt either from explicit instruction or by trialand-error. We report that the networks activate en masse and in synchrony when novel rules are being learnt from instruction. However, these networks are not homogeneous in their functions; instead, the directed connectivities between them vary asymmetrically across the learning timecourse and they disengage from the task sequentially along a rostro-caudal axis. Furthermore, when negative feedback indicates the need to switch to alternative stimulusresponse rules, there is additional input to the LFC networks from the ventral striatum. These results support the hypotheses that LFC networks interact as a hierarchical system during intentional learning and that signals from the ventral striatum have a driving influence on this system when the internal program for processing the task is updated.
Question Answering on SQuAD 2.0 using QANet with Performer FastAttention

This project reproduced the QANet model using Performer Fast Attention for the SQuAD 2.0 eading comprehension task. Fast Attention approximates self-attention in linear complexity, provable and tuneable accuracy, making Transformers suitable for large-scale tasks. We substituted self-attention layers with Fast Attention in QANet encoder blocks and evaluated the model with different feature counts, gaining 18% faster training and keeping accuracy
The Black Swan Problem: The Role of Capital, Liquidity and Operating Flexibility

How firms cope with tail risk is an under researched problem in the literature on corporate risk management. This paper presents stylized facts on the nature of revenue shocks based on 65 years worth of Compustat data. We define a Black Swan as an unexpected year-on-year drop in revenue between 30-90%. The rate of Black Swans has increased markedly since the 1970s and there are more pronounced cyclical peaks in the three most recent decades. We also examine the role of three general determinants of firms’ ability to absorb Black Swans: equity capital, liquidity, and operating flexibility. The conclusion to emerge from this analysis is that the deciding factor in mediating the effects of revenue shocks on employment is liquidity. Cash reserves and cash margins make firms less fragile, but neither equity capital nor operating flexibility robustly buffer against Black Swans.
Be Considerate: Objectives, Side Effects, and Deciding How to Act

Recent work in AI safety has highlighted that in sequential decision making, objectives are often underspecified or incomplete. This gives discretion to the acting agent to realize the stated objective in ways that may result in undesirable outcomes. We contend that to learn to act safely, a reinforcement learning (RL) agent should include contemplation of the impact of its actions on the wellbeing and agency of others in the environment, including other acting agents and reactive processes. We endow RL agents with the ability to contemplate such impact by augmenting their reward based on expectation of future return by others in the environment, providing different criteria for characterizing impact. We further endow these agents with the ability to differentially factor this impact into their decision making, manifesting behavior that ranges from self-centred to self-less, as demonstrated by experiments in gridworld environments.