Miracle-dz/UniFAR-FTUs huggingface dataset:
Physiological Profile of Neuropeptide Y-Expressing Neurons in Bed Nucleus of Stria Terminalis in Mice: State of High Excitability

Both, the anterior bed nucleus of the stria terminalis (BNST) and the neuropeptide Y (NPY) system are involved in shaping fear and defensive responses that adapt the organism to potentially life-threatening conditions. NPY is expressed in the BNST but NPY-expressing neurons in this critical hub in the stress response network have not been addressed before. Therefore, we performed whole-cell patch-clamp recordings in acute slices of anterior BNST from Npy-hrGFP transgenic mice to identify and characterize NPY-expressing neurons. We show that NPY-positive and NPY-negative neurons in anterior BNST match the previous classification scheme of type I (Regular Spiking), type II (Low-Threshold Bursting), and type III (fast Inward Rectifying) cells, although the proportion of these physiological phenotypes was similar within both neuronal subpopulations. However, NPY-positive and NPY-negative neurons possessed distinct intrinsic electrophysiological properties. NPY-positive neurons displayed higher input resistance and lower membrane capacitance, corresponding to small cell bodies and shorter less ramified dendrites, as compared to their NPY-negative counterparts. Furthermore, NPY-positive neurons generated higher frequent series of action potentials upon membrane depolarization and displayed significantly lower GABAA receptor-mediated synaptic responsiveness during evoked, spontaneous, and elementary synaptic activity. Taken together, these properties indicate an overall state of high excitability in NPY-positive neurons in anterior BNST. In view of the role of the anterior BNST in anxiety- and stress-related behaviors, these findings suggest a scenario where NPY-positive neurons are preferentially active and responsive to afferent inputs, thereby contributing to adaptation of the organism to stressful environmental encounters.
Analysing Dropout and Compounding Errors in Neural Language Models

This paper carries out an empirical analysis of various dropout techniques for language modelling, such as Bernoulli dropout, Gaussian dropout, Curriculum Dropout, Variational Dropout and Concrete Dropout. Moreover, we propose an extension of variational dropout to concrete dropout and curriculum dropout with varying schedules. We find these extensions to perform well when compared to standard dropout approaches, particularly variational curriculum dropout with a linear schedule. Largest performance increases are made when applying dropout on the decoder layer. Lastly, we analyze where most of the errors occur at test time as a post-analysis step to determine if the well-known problem of compounding errors is apparent and to what end do the proposed methods mitigate this issue for each dataset. We report results on a 2-hidden layer LSTM, GRU and Highway network with embedding dropout, dropout on the gated hidden layers and the output projection layer for each model. We report our results on Penn-TreeBank and WikiText-2 word-level language modelling datasets, where the former reduces the long-tail distribution through preprocessing and one which preserves rare words in the training and test set.
Zero-Shot Transfer VQA Dataset

Acquiring a large vocabulary is an important aspect of human intelligence. Onecommon approach for human to populating vocabulary is to learn words duringreading or listening, and then use them in writing or speaking. This ability totransfer from input to output is natural for human, but it is difficult for machines.Human spontaneously performs this knowledge transfer in complicated multimodaltasks, such as Visual Question Answering (VQA). In order to approach human-levelArtificial Intelligence, we hope to equip machines with such ability. Therefore, toaccelerate this research, we propose a newzero-shot transfer VQA(ZST-VQA)dataset by reorganizing the existing VQA v1.0 dataset in the way that duringtraining, some words appear only in one module (i.e. questions) but not in theother (i.e. answers). In this setting, an intelligent model should understand andlearn the concepts from one module (i.e. questions), and at test time, transfer themto the other (i.e. predict the concepts as answers). We conduct evaluation on thisnew dataset using three existing state-of-the-art VQA neural models. Experimentalresults show a significant drop in performance on this dataset, indicating existingmethods do not address the zero-shot transfer problem. Besides, our analysis findsthat this may be caused by the implicit bias learned during training.
Automated Theorem Proving in Intuitionistic Propositional Logic by Deep Reinforcement Learning

The problem-solving in automated theorem proving (ATP) can be interpreted as a search problem where the prover constructs a proof tree step by step. In this paper, we propose a deep reinforcement learning algorithm for proof search in intuitionistic propositional logic. The most significant challenge in the application of deep learning to the ATP is the absence of large, public theorem database. We, however, overcame this issue by applying a novel data augmentation procedure at each iteration of the reinforcement learning. We also improve the efficiency of the algorithm by representing the syntactic structure of formulas by a novel compact graph representation. Using the large volume of augmented data, we train highly accurate graph neural networks that approximate the value function for the set of the syntactic structures of formulas. Our method is also cost-efficient in terms of computational time. We will show that our prover outperforms Coq's $\texttt{tauto}$ tactic, a prover based on human-engineered heuristics. Within the specified time limit, our prover solved 84% of the theorems in a benchmark library, while $\texttt{tauto}$ was able to solve only 52%.
Convolutional Neural Networks-Based MRI Image Analysis for the Alzheimer’s Disease Prediction From Mild Cognitive Impairment

Mild cognitive impairment (MCI) is the prodromal stage of Alzheimer’s disease (AD). Identifying MCI subjects who are at high risk of converting to AD is crucial for effective treatments. In this study, a deep learning approach based on convolutional neural networks (CNN), is designed to accurately predict MCI-to-AD conversion with magnetic resonance imaging (MRI) data. First, MRI images are prepared with age-correction and other processing. Second, local patches, which are assembled into 2.5 dimensions, are extracted from these images. Then, the patches from AD and normal controls (NC) are used to train a CNN to identify deep learning features of MCI subjects. After that, structural brain image features are mined with FreeSurfer to assist CNN. Finally, both types of features are fed into an extreme learning machine classifier to predict the AD conversion. The proposed approach is validated on the standardized MRI datasets from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) project. This approach achieves an accuracy of 79.9% and an area under the receiver operating characteristic curve (AUC) of 86.1% in leave-one-out cross validations. Compared with other state-of-the-art methods, the proposed one outperforms others with higher accuracy and AUC, while keeping a good balance between the sensitivity and specificity. Results demonstrate great potentials of the proposed CNN-based approach for the prediction of MCI-to-AD conversion with solely MRI data. Age correction and assisted structural brain image features can boost the prediction performance of CNN.
A Stronger Baseline for Multilingual Word Embeddings

Levy, Sogaard and Goldberg’s (2017) S-ID (sentence ID) method applies word2vec on tuples containing a sentence ID and a word from the sentence. It has been shown to be a strong baseline for learning multilingual embeddings. Inspired by recent work on concept based embedding learning we propose SC-ID, an extension to S-ID: given a sentence aligned corpus, we use sampling to extract concepts that are then processed in the same manner as S-IDs. We perform experiments on the Parallel Bible Corpus across 1000+ languages and show that SC-ID yields up to 6% performance increase in a word translation task. In ad- dition, we provide evidence that SC-ID is easily and widely applicable by reporting competitive results across 8 tasks on a EuroParl based corpus.
Essays in Quantitative Risk Management for Financial Regulation of Operational Risk Models

An extensive amount of evolving guidance and rules are provided to banks by financial regulators. A particular set of instructions outline requirements to calculate and set aside loss-absorbing regulatory capital to ensure the solvency of a bank. Mathematical models are typically used by banks to quantify sufficient amounts of capital. In this thesis, we explore areas that advance our knowledge in regulatory risk management. In the first essay, we explore an aspect of operational risk loss modeling using scenario analysis. An actuarial modeling method is typically used to quantify a baseline capital value which is then layered with a judgemental component in order to account for and integrate what-if future potential losses into the model. We propose a method from digital signal processing using the convolution operator that views the problem of the blending of two signals. That is, a baseline loss distribution obtained from the modeling of frequency and severity of internal losses is combined with a probability distribution obtained from scenario responses to yield a final output that integrates both sets of information. In the second essay, we revisit scenario analysis and the potential impact of catastrophic events to that of the enterprise level of a bank. We generalize an algorithm to account for multiple level of intensities of events together with unique loss profiles depending on the business units effected.
Building Intelligent Digitial Assistants for Speakers of a Lesser-Resourced Language

This paper reports on the work to develop intelligent digital assistants for speakers of a lesser-resourced language, namely Welsh. Such assistants provided by commercial vendors such as Apple (Siri), Amazon (Alexa), Microsoft (Cortana) and Google (Google Now) are allowing users increasingly to speak in natural English with their devices and computers in order to complete tasks, obtain assistance and request information. We demonstrate how these systems’ architectures do not provide the means for external developers to build intelligent speech interfaces for additional languages, which, in the case of less resourced languages, are likely to remain unsupported. Consequently we document how such an obstacle has been tackled with open alternatives. The paper highlights how previous work on Welsh language speech recognition were improved, utilized and integrated into an existing open source intelligent digital assistant software project. The paper discusses how this work hopes to stimulate further developments and include Welsh and other lesserresourced languages in as many developments of intelligent digital assistants as possible.
Addressing Opioid-Related Chemical Coping in Long-Term Opioid Therapy for Chronic Noncancer Pain: A Multicenter, Observational, Cross-Sectional Study

Opioid consumption has increased worldwide, which carries the risk of opioid use disorder (OUD). However, the literature on OUD and opioid-related chemical coping (OrCC) in chronic noncancer pain (CNCP) is heterogeneous, with most studies conducted in the United States. We performed a multicenter, observational, cross-sectional study to address OrCC in long-term opioid therapy (LtOT) for CNCP in South Korea. The objectives were to determine the frequency and predictors of OrCC. We included 258 patients. Among them, fifty-five (21%) patients showed OrCC. The sample had high pain catastrophizing (≥30 points; 66%), moderate-severe insomnia (≥15 points; 63%), low resilience (68 points), and high suicidal ideation (67%). OrCC patients had greater pain interference (85.18% vs. 58.28%, p = 0.017) and lower satisfaction with the LtOT (56.4% vs. 78.3%, p = 0.002). In multivariable analysis, alcohol abuse (OR = 6.84, p = 0.001), prescription drugs abuse (OR = 19.32, p = 0.016), functional pain (OR = 12.96, p < 0.001), head and neck pain (OR = 2.48, p = 0.039), MEDD (morphine equivalent daily dose) ≥ 200 mg/day (OR = 3.48, p = 0.006), and ongoing litigation (OR = 2.33, p = 0.047) were significant predictors of OrCC. In conclusion, the break-out of OrCC in CNCP in South Korea was comparable to those in countries with high opioid consumption, such as the United States, regardless of the country’s opioid consumption rate.
A Fast Algorithm for Clustering High Dimensional Feature Vectors

We propose an algorithm for clustering high dimensional data. If $P$ features for $N$ objects are represented in an $N\times P$ matrix ${\bf X}$, where $N\ll P$, the method is based on exploiting the cluster-dependent structure of the $N\times N$ matrix ${\bf XX}^T$. Computational burden thus depends primarily on $N$, the number of objects to be clustered, rather than $P$, the number of features that are measured. This makes the method particularly useful in high dimensional settings, where it is substantially faster than a number of other popular clustering algorithms. Aside from an upper bound on the number of potential clusters, the method is independent of tuning parameters. When compared to $16$ other clustering algorithms on $32$ genomic datasets with gold standards, we show that it provides the most accurate cluster configuration more than twice as often than its closest competitors. We illustrate the method on data taken from highly cited genomic studies.
Interplay Between MicroRNAs and Targeted Genes in Cellular Homeostasis of Adult Zebrafish (Danio rerio)

Background: Cellular homeostasis is regulated by the intricate interplay between a plethora of signaling pathways and “energetic sensors” in organs. In order to maintain energy balance, induction or repression of metabolic pathways must be regulated and act in concert with the energetic demands of the cell at a given point in time. A new class of small noncoding RNAs, the microRNAs (miRNAs), has added yet further complexity to the control of metabolic homeostasis. Objective: Understanding the damages induced by toxins in the liver and the intestine as well as the interplay between the miRNome and transcriptome first requires baseline characterization in these tissues in healthy animals under cellular homeostasis. Methods: The liver (main site for detoxification) and the gut (primary exposure routes for contaminant exposure) were dissected out (wildtype fish), total and small RNA extracted, mRNA and miRNA libraries constructed and subjected to high throughput sequencing. Differential Expression (DE) analysis was performed comparing liver with gut and an “miRNA matrix” that integrates the miRNA-seq and mRNA-seq data was constructed. Results: Both the miRNome and transcriptome of the liver and gut tissues were characterized and putative novel miRNAs were identified. Exploration of the “miRNA matrix” regulatory network revealed that miRNAs uniquely expressed in the liver or gut tissue regulated fundamental cellular processes important for both organs, and that commonly expressed miRNAs in both tissues regulated biological processes that were specific to either the liver or the gut. Conclusion: The result of our analyses revealed new insights into microRNA function in these tissues.
Bayesian Ensembles of Crowds and Deep Learners for Sequence Tagging

Current methods for sequence tagging, a core task in NLP, are data hungry. Crowdsourcing is a relatively cheap way to obtain labeled data, but the annotators are unreliable. To address this, we develop a modular Bayesian method for aggregating sequence labels from multiple annotators and evaluate different models of annotator errors and labeling biases. Our approach integrates black-box sequence taggers as components in the model to improve the quality of predictions. We evaluate our model on crowdsourced data for named entity recognition and information extraction tasks, showing that our sequential annotator model outperforms previous methods.
Importance of Seismic Diffractions for Fractures Imaging

The use of diffraction imaging is important and rapidly gaining momentum in the oil and gas industry as the need of the industry moves toward exploiting smaller and more complex structures to find hydrocarbon. Illumination of these small scale discrete transmissivity structures such as faults or fracture zones prior to exploration offers substantial benefits for all phases of field development. We have established a new way of seismic imaging through diffractions study using reflection seismology. A subsurface geological model is developed from one of highly productive field located in the southern part of the Malay Basin. To study the seismic component in reflection seismology, a Finite difference modelling is used to generate synthetic seismic data. Both velocity and density models are used for the explanation of wave propagation, intimating the subsurface from side to side in both one-way and two-wave wave propagation. One–way wave equation migration methods are tested to image the faults in the synthetic seismic section of the Malay basin and we demonstrated the need of diffraction imaging for highly dipping faults and complex structure.
Training Toward a Movement: Career Development Insights From the First 7 Years of a Global Health Equity Residency.

Background The Doris and Howard Hiatt Residency in Global Health Equity and Internal Medicine at Brigham and Women's Hospital provides global health training during residency, but little is known about its effect on participants' selection of a global health career. Objective We assessed the perceptions of residency graduates from the first 7 classes to better understand the outcomes of this education program, and the challenges faced by participants. Methods We interviewed 27 of 31 physicians (87%) who graduated from the program between 2003 and 2013 using a convergent mixed-methods design and a structured interview tool that included both open-ended and forced-choice questions. We independently coded and analyzed qualitative data using a case study design, and then wove together the qualitative and quantitative data at the interpretation phase using a parallel convergent mixed-methods design. Results Entering a career focused on social justice was cited as the most common motivator for selecting to train in global health. Most respondents (83%, 20 of 24) reported they were able to achieve this goal despite structural barriers, such as lower salaries compared with peers, a lack of mentors in the field, poorly structured and undersupported career pathways at their institutions, and unique work-life challenges. Conclusions A majority of graduates from 1 dedicated residency program in global health and internal medicine reported they were able to continue to engage in global health activities after graduation and, despite identified challenges, reported that they planned long-term careers in global health.
Implicit Regularization of Stochastic Gradient Descent in Natural Language Processing: Observations and Implications

Deep neural networks with remarkably strong generalization performances are usually over-parameterized. Despite explicit regularization strategies are used for practitioners to avoid over-fitting, the impacts are often small. Some theoretical studies have analyzed the implicit regularization effect of stochastic gradient descent (SGD) on simple machine learning models with certain assumptions. However, how it behaves practically in state-of-the-art models and real-world datasets is still unknown. To bridge this gap, we study the role of SGD implicit regularization in deep learning systems. We show pure SGD tends to converge to minimas that have better generalization performances in multiple natural language processing (NLP) tasks. This phenomenon coexists with dropout, an explicit regularizer. In addition, neural network's finite learning capability does not impact the intrinsic nature of SGD's implicit regularization effect. Specifically, under limited training samples or with certain corrupted labels, the implicit regularization effect remains strong. We further analyze the stability by varying the weight initialization range. We corroborate these experimental findings with a decision boundary visualization using a 3-layer neural network for interpretation. Altogether, our work enables a deepened understanding on how implicit regularization affects the deep learning model and sheds light on the future study of the over-parameterized model's generalization ability.
Leadership Observation and Feedback Tool: A Novel Instrument for Assessment of Clinical Leadership Skills.

Background While leadership training is increasingly incorporated into residency education, existing assessment tools to provide feedback on leadership skills are only applicable in limited contexts. Objective We developed an instrument, the Leadership Observation and Feedback Tool (LOFT), for assessing clinical leadership. Methods We used an iterative process to develop the tool, beginning with adapting the Leadership Practices Inventory to create an open-ended survey for identification of clinical leadership behaviors. We presented these to leadership experts who defined essential behaviors through a modified Delphi approach. In May 2014 we tested the resulting 29-item tool among residents in the internal medicine and pediatrics departments at 2 academic medical centers. We analyzed instrument performance using Cronbach's alpha, interrater reliability using intraclass correlation coefficients (ICCs), and item performance using linear-by-linear test comparisons of responses by postgraduate year, site, and specialty. Results A total of 377 (of 526, 72%) team members completed the LOFT for 95 (of 519, 18%) residents. Overall ratings were high-only 14% scored at the novice level. Cronbach's alpha was 0.79, and the ICC ranged from 0.20 to 0.79. Linear-by-linear test comparisons revealed significant differences between postgraduate year groups for some items, but no significant differences by site or specialty. Acceptability and usefulness ratings by respondents were high. Conclusions Despite a rigorous approach to instrument design, we were unable to collect convincing validity evidence for our instrument. The tool may still have some usefulness for providing formative feedback to residents on their clinical leadership skills.