Miracle-dz/UniFAR-FTUs huggingface dataset:
Investigating QANet’s Convolution Layer

Tasked with building a model for Question-Answering on the SQuAD 2.0 dataset, we re-implement QANet and perform several extensions to boost performance. The motivation of QANet is that it replaces the RNN in traditional QA models (like BiDAF) with convolutions, which can lead to faster training and inference. Our further experiments investigate the purpose of QANet’s convolution layer by assessing performance when swapping it with local and global self-attention layers. This contributes to an understanding of whether the convolution layer does indeed capture local structure in sentences, as the original authors of QANet posited. We observed small improvements by changing the number of blocks in each layer of model encoders, and our best performance was achieved when creating an ensemble of results from QANet models of various model encoder layer block sizes (+8.4 F1 and +8.6 EM compared to BiDAF baseline). The experimental model that used repeated local self-attention layers instead of convolution layers outperformed the model that replaced the convolution layers with global self-attention layers, supporting the original authors’ hypothesis that the convolution layer functions primarily to capture local structure.
SMU Data Science Review SMU Data

Automated speaker verification has been an area of increased research in the last few years, with a special interest in metric learning approaches that compute distances between speaker voiceprints. In this paper, three metric learning systems are built and compared in a one-shot speaker verification task using contrastive max-margin loss, triplet loss, and quadruplet loss. For all the models, spectrograms are created from speaker audio. Convolutional Neural Network embedding layers are trained to produce compact voiceprints that allow users to be distinguished using distance calculations. Performances of the three models were similar, but the model with the best EER used triplet loss in this experiment.
Building a Robust QA system

In this project, we aim to build a robust question answering system that can adapt to out-of-domain datasets. A single network may overfit to the superficial correlation in the training distribution, but with a meaningful number of expert sub-networks, a gating network that selects a sparse combination of experts for each input, and careful balance on the importance of expert sub-networks, the Mixture-of-Experts (MoE) model allows us to train a multi-task learner that can be generalized to out-of-domain datasets. We also explore the possibility of bringing the MoE layers up to the middle of the DistilBERT [1] and replacing the dense feed forward network with a sparsely-activated switch FFN layers, similar to the Switch Transformer [2] architecture, which simplifies the MoE routing algorithm with reduced communication and computational costs. In addition to model architectures, we explore techniques of data augmentation including Easy Data Augmentation (EDA) and back translation, to create more meaningful variance among the small out-of-domain training data, therefore boost the performance and robustness of our models. In this report we show that our combination of best architecture and data augmentation techniques achieves a 53.477 F1 score in the out-of-domain evaluation, which is a 9.52% performance gain over the baseline. On the final test set, we reported a higher 59.506 F1 and 41.651 EM. We successfully demonstrate the effectiveness of Mixture-of-Expert architecture in a Robust QA task.
Novel Visual-Textual Attention Fusion Models for Context-Aware Image Alt Text Generation

Images are indispensable to web communication but are often inaccessible to individuals with blind or low vision. Screen readers use descriptions embedded in the HTML alt tag, which is often missing in images; while captions are more plentiful, they are written to supplement the image with context rather than replace it—and are typically devoid of visual details requisite for accessibility. Furthermore, existing description generation systems are often generic and agnostic to contextual implicature. To address this issue, we introduce an image description model using a novel multi-modal architecture that fuses visual embeddings from an image input and text embeddings from a caption context input. The description is generated using a GRU recurrent neural network (RNN) decoder, with Bahdanau attention units applied over tensors from both spatial image patches and caption tokens to encourage the RNN to extract from both visual and textual knowledge. We also develop a novel beam search variant with a brevity penalty to generate thorough descriptions. We demonstrate that our multi-modal method outperforms unimodal models with state-of-the-art architectures that do not use context input in BLEU score on the Wikipedia-based Concadia dataset. Our work represents a contribution toward enriching visual accessibility through context-aware NLP systems.
Neuroeconomics, decision-making and rationality

Les outils des neurosciences nous permettent d’aborder le probleme de la prise de decision et peuvent notamment eclairer la rationalite des choix hedoniques qu’il s’agisse de decision de consommation, de decisions en interaction sociale, de choix intertemporels, de decisions dans le risque ou l’incertitude. Les neurosciences permettent en effet de distinguer rationalite interne et rationalite externe Ainsi, il semble que si les choix sont motives par des considerations d’ordre emotionnel, notamment le gout pour la recompense mais egalement l’aversion a la douleur ou a la punition, la decision resulte souvent de trade-offs neuronaux qui semblent s’operer au moment du choix et qui caracteriseraient une rationalite interne par opposition a la rationalite externe proche d’une rationalite substantive. Il ressort que si la deliberation ne constitue pas une condition suffisante a une rationalite des actes, la rationalite des processus pourrait, elle, s’inscrire dans une perspective evolutionniste.
ExtraPhraseRank: A Length-Controlled Data Augmentation Strategy for Unsupervised Abstractive Summarization

The emergence of large pre-trained language models has redefined the state of the art for abstractive summarization. Unfortunately, the lack of parallel data for this task is a hindrance to the usefulness of these models. Further, existing methods of unsupervised summarization often lack control of their output length. We solve these problems by presenting ExtraPhraseRank, a data augmentation strategy for the efficient, length-controlled generation of synthetic summaries. The algorithm has two major steps: it first invokes the TextRank algorithm [1] to extract the most important sentences in a text. It then back-translates the most important sentences (by translating them to German, and back to English) in order to increase the diversity of words and phrases. We test this method on a popular abstractive summarization dataset, and then use the generated summaries to fine-tune a BART model [2] for the document summarization task. We find that this strategy produces modest improvements in ROUGE scores above our base-lines, but that it is ultimately not as powerful as fine-tuning even a smaller number of human-written summaries. Finally, we test the practicality of our algorithm on a multi-document summarization task with no existing labels (namely, summarizing collecions of amazon reviews). We find that, though promising in the way of controlling the output length of summaries generated, the algorithm does not lead to summaries significantly more useful than a vanilla BART model.
Murazvu, George V Software Testing: An Analysis of the Impacts of Test Automation on Software’s Cost, Quality and Time

The University Repository is a digital collection of the research output of the University, available on Open Access. Copyright and Moral Rights for the items on this site are retained by the individual author and/or other copyright owners. Users may access full items free of charge; copies of full text items generally can be reproduced, displayed or performed and given to third parties in any format or medium for personal research or study, educational or not-for-profit purposes without prior permission or charge, provided:
Lexical transfer in Norwegian interlanguage : A detection-based approach

This study investigates cross-linguistic influence (‘transfer’) in Norwegian interlanguage using predictive data mining technology with a focus on lexical transfer. The impetus for the present work came from the publication of a series of studies (Jarvis & Crossley 2012) that explore the ‘detection-based approach’ to language transfer. The following research questions are addressed: 1. Can data mining techniques be used to identify the L1 background of Norwegian language learners on the basis of their use of lexical features of the target language? 2. If so, what are the best predictors of L1 background? 3. And can those predictors be traced to cross-linguistic influence? The study utilizes data from Norsk andrespråkskorpus (ASK), the Norwegian Second Language Corpus housed at the University of Bergen, and draws on resources from the ASKeladden project. The source data consists of texts written by 1,736 second language learners of Norwegian from ten different L1 backgrounds, and a control corpus of 200 texts written by native speakers. Word frequencies computed from this data are analysed using multivariate statistical methods that include analysis of variance and linear discriminant analysis, and the results are subjected to contrastive analysis. The combination of discriminant analysis and contrastive analysis produces all three types of evidence called for by Jarvis (2000) in his methodological requirements for language transfer research: intragroup homogeneity, intergroup heterogeneity and cross-language congruity. Well-known transfer effects, such as the tendency for Russian learners to omit indefinite articles, are confirmed, and other, more subtle patterns of learner language are revealed, such as the tendency amongst Dutch learners to overuse the modal verb skal to a far greater extent than other learners. In addition to confirming the reality of lexical transfer, these results provide abundant material for further research, while the methodology employed can be harnessed in many areas of linguistic research. ii Lexical transfer in Norwegian interlanguage
Data Augmentation with Adversarial Examples and Back-Translation

Though question answering (QA) systems serve essential roles online and drive the websites of many Internet-based companies, they often perform poorly because of NLP models’ inability to learn deeply beyond their given training distribution. Here, we seek to improve upon basic QA models by implementing a robust QA system that can generalize to unseen, out-of-domain data. Our QA model is based on data augmentation, primarily with adversarial examples generated by a language model replacing one masked token in our original data, and backtranslation using Russian as a pivot language. Compared to the DistilBERT baseline dev F1 and EM of 47.72 and 30.63 respectively, our best data augmentation-based model achieved dev F1 and EM of 49.07 and 31.68, and test F1 and EM of 59.48 and 40.51.
1q21.1 deletion and a rare functional polymorphism in siblings with thrombocytopenia-absent radius–like phenotypes

Thrombocytopenia-absent radii (TAR) syndrome, characterized by neonatal thrombocytopenia and bilateral radial aplasia with thumbs present, is typically caused by the inheritance of a 1q21.1 deletion and a single-nucelotide polymorphism in RBM8A on the nondeleted allele. We evaluated two siblings with TAR-like dysmorphology but lacking thrombocytopenia in infancy. Family NCI-107 participated in an IRB-approved cohort study and underwent comprehensive clinical and genomic evaluations, including aCGH, whole-exome, whole-genome, and targeted sequencing. Gene expression assays and electromobility shift assays (EMSAs) were performed to evaluate the variant of interest. The previously identified TAR-associated 1q21.1 deletion was present in the affected siblings and one healthy parent. Multiple sequencing approaches did not identify previously described TAR-associated SNPs or mutations in relevant genes. We discovered rs61746197 A > G heterozygosity in the parent without the deletion and apparent hemizygosity in both siblings. rs61746197 A > G overlaps a RelA–p65 binding motif, and EMSAs indicate the A allele has higher transcription factor binding efficiency than the G allele. Stimulation of K562 cells to induce megakaryocyte differentiation abrogated the shift of both reference and alternative probes. The 1q21.1 TAR-associated deletion in combination with the G variant of rs61746197 on the nondeleted allele is associated with a TAR-like phenotype. rs61746197 G could be a functional enhancer/repressor element, but more studies are required to identify the specific factor(s) responsible. Overall, our findings suggest a role of rs61746197 A > G and human disease in the setting of a 1q21.1 deletion on the other chromosome.
Explore Different Transfer Learning Methods for More Robust Q&A Models

When given a large amount of data, Natural Language Processing (NLP) systems that are fine-tuned on pretrained language models are able to achieve good performance in Question Answering (QA) task. Yet, these systems cannot generalized well to datasets from unseen domains. Kumar et al. [1] introduce the Feature Distortion Theory, which attempts to explain the poor performance of the fully fine-tuned pretrained model on out-of-domain image classification tasks. The theory suggests that complete fine-tuning distorts the pretrained features. In this project, we test the theory in NLP domain for QA task. Using the pretrained Distil-BERT model and applying different partial fine-tuning strategies before fine-tuning the full model, we find that partial fine-tuning does not significantly improve the performance. In addition, to enhance the robustness of the QA system, we also use other out-of-domain adaptation and few-shots learning methods, including Data Augmentation and Mixture of Experts. The best model achieves F1 = 60.49 and EM = 42.5 on the out-of-domain test set.
Scale-Dependence of Hematite Nanoparticle Sulfidation Scale-Dependence of Hematite Nanoparticle Sulfidation

Iron (oxyhydr)oxides are widespread in the environment and have a disproportionate effect on the fate of metal(loid)s in groundwater. Numerous studies demonstrate the effect of iron-sulfur dynamics on contaminant sequestration, and particle size further impacts the reactivity of iron (oxyhydr)oxides, including their capacity for sorption and precipitation. This study examines the role of particle size on the sulfidation of iron (oxyhydr)oxides. Synthetic hematite nanoparticles with approximate diameters of 7.1 nm and 104 nm were coated onto quartz sand and reacted with bisulfide by advective flow. Characterization of these nanoparticles revealed hydroxyls present at the hematite surface, with large particles having a higher hydroxide content per unit of surface area. Analysis of effluent from advective flow experiments show that the hematite nanoparticles undergo reductive dissolution of iron followed by precipitation of FeS. Iron concentrations increased with increasing bisulfide concentration, and ferrous iron in the effluent was higher for small particles than large particles when eluted with 10 mM bisulfide and lower with 0.1 and 1 mM bisulfide. Mineral transformations indicative of FeS precipitation occurred at 1 mM for small particles and at 10 mM for both particle sizes. studies previously do not examine the sulfidation of iron (oxyhydr)oxide nanoparticles under continuous flow for minerals aside from ferrihydrite. This study investigates the role of particle size on the reductive dissolution of iron oxide nanoparticles of two different sizes by bisulfide, a product of microbial sulfate reduction and a strong reductant, under dynamic flow conditions. Hematite is the most stable iron (oxyhydr)oxide bulk material under ambient conditions and exhibits large differences in its physiochemical and thermodynamic properties at the nanoscale. Column experiments consist of nanoparticle hematite-coated packed sands exposed to varying concentrations of bisulfide under anoxic, advective flow conditions. Analyses include quantification of dissolved iron and bisulfide concentrations in the column effluent. An integrated kinetic model is developed based on effluent data to determine reaction mechanism and rate constants as a function of particle size. Understanding the role of size on the reactivity and stability of iron (oxyhydr)oxide minerals is important to consider in the fate and transport of metals and metalloids in groundwater where nanoparticles are present.
Rho-associated kinase and zipper-interacting protein kinase, but not myosin light chain kinase, are involved in the regulation of myosin phosphorylation in serum-stimulated human arterial smooth muscle cells

Myosin regulatory light chain (LC20) phosphorylation plays an important role in vascular smooth muscle contraction and cell migration. Ca2+/calmodulin-dependent myosin light chain kinase (MLCK) phosphorylates LC20 (its only known substrate) exclusively at S19. Rho-associated kinase (ROCK) and zipper-interacting protein kinase (ZIPK) have been implicated in the regulation of LC20 phosphorylation via direct phosphorylation of LC20 at T18 and S19 and indirectly via phosphorylation of MYPT1 (the myosin targeting subunit of myosin light chain phosphatase, MLCP) and Par-4 (prostate-apoptosis response-4). Phosphorylation of MYPT1 at T696 and T853 inhibits MLCP activity whereas phosphorylation of Par-4 at T163 disrupts its interaction with MYPT1, exposing the sites of phosphorylation in MYPT1 and leading to MLCP inhibition. To evaluate the roles of MLCK, ROCK and ZIPK in these phosphorylation events, we investigated the time courses of phosphorylation of LC20, MYPT1 and Par-4 in serum-stimulated human vascular smooth muscle cells (from coronary and umbilical arteries), and examined the effects of siRNA-mediated MLCK, ROCK and ZIPK knockdown and pharmacological inhibition on these phosphorylation events. Serum stimulation induced rapid phosphorylation of LC20 at T18 and S19, MYPT1 at T696 and T853, and Par-4 at T163, peaking within 30–120 s. MLCK knockdown or inhibition, or Ca2+ chelation with EGTA, had no effect on serum-induced LC20 phosphorylation. ROCK knockdown decreased the levels of phosphorylation of LC20 at T18 and S19, of MYPT1 at T696 and T853, and of Par-4 at T163, whereas ZIPK knockdown decreased LC20 diphosphorylation, but increased phosphorylation of MYPT1 at T696 and T853 and of Par-4 at T163. ROCK inhibition with GSK429286A reduced serum-induced phosphorylation of LC20 at T18 and S19, MYPT1 at T853 and Par-4 at T163, while ZIPK inhibition by HS38 reduced only LC20 diphosphorylation. We also demonstrated that serum stimulation induced phosphorylation (activation) of ZIPK, which was inhibited by ROCK and ZIPK down-regulation and inhibition. Finally, basal phosphorylation of LC20 in the absence of serum stimulation was unaffected by MLCK, ROCK or ZIPK knockdown or inhibition. We conclude that: (i) serum stimulation of cultured human arterial smooth muscle cells results in rapid phosphorylation of LC20, MYPT1, Par-4 and ZIPK, in contrast to the slower phosphorylation of kinases and other proteins involved in other signaling pathways (Akt, ERK1/2, p38 MAPK and HSP27), (ii) ROCK and ZIPK, but not MLCK, are involved in serum-induced phosphorylation of LC20, (iii) ROCK, but not ZIPK, directly phosphorylates MYPT1 at T853 and Par-4 at T163 in response to serum stimulation, (iv) ZIPK phosphorylation is enhanced by serum stimulation and involves phosphorylation by ROCK and autophosphorylation, and (v) basal phosphorylation of LC20 under serum-free conditions is not attributable to MLCK, ROCK or ZIPK.
Understanding Attention for Question Answering models

In this work, we explore and compare the performances of non Pre-trained Contex-tual Embeddings (PCE) based approaches - Bidirectional Attention Flow (BiDAF), Dynamic Coattention Network (DCN), and FusionNet and also consider ensemble based approaches of FusionNet to get an idea of attention focused models’ effectiveness on the SQuAD dataset[1]. We aim to reproduce the papers by implementing the network architectures using the PyTorch[2] framework. Reproducing the DCN paper, we obtain an F1 score of 49.67% on the Stanford Question Answering Dataset (SQuAD2.0). We obtain an F1 score of 64.49% using the FusionNet single model.
Occupational Safety and Health Coordinators – Puzzle-piece Caretakers or Necessary Evils 1

The construction industry continues to be high-risk in terms of occupational safety and health (OSH) issues. A pivotal instrument in preventing these risks at both European and Danish levels is the OSH coordinator. In spite of the important role of the coordinator, little research on their roles and functions exist, and critics have pointed out that OSH professionals in general may only confer limited impact on preventive OSH work. This study argues that professional identities and struggles to maintain preferred, as well as rejecting unwanted identities are highly important to understand OSH coordinators’ practices. The study investigates OSH coordinators professional identities and their implications for practice through analysis of interviews with 12 experienced OSH coordinators in the Danish construction industry. The study reveals how struggles for maintaining a positive self-image and social recognition may explain why coordinators struggle to prioritize preventing OSH risks over legitimization and social practices.
ASSESSING BIOINSPIRED CONCEPTS FOR SPACE DEBRIS REMOVAL AND EVALUATING THEIR FEASIBILITY FOR SIMPLE DEMONSTRATOR DESIGN

The ESA-funded project BIOINSPACED intends to ﬁnd biomimetic solutions for new technologies that can contribute to ESA’s ’Clean Space’ initiative by mitigating space debris. Analyzing existing biomimetic examples and screening nature’s idea pool helps deﬁning new bioinspired solutions that have the potential to fulﬁl the technical requirements related to an active debris removal mission. This paper will expand the current state of the art of active debris removal by describing a newly developed catalogue of 130 collected concepts with potential for future debris removal missions. A feasibility analysis was conducted, evaluating and subsequently enabling clustering and ranking of different concepts regarding their functionality and applicability. The ten most promising principles are presented in this study, and will be used to conceptualize holistic debris removal scenarios and manufacture one simple demonstrator during the next phase of the project, harvesting and illustrating their biomimetic potential for innovative solutions.