ymoslem/acl-abstracts huggingface dataset:
Contextualized French Language Models for Biomedical Named Entity Recognition (2020)

Named entity recognition (NER) is key for biomedical applications as it allows knowledge discovery in free text data. As entities are semantic phrases, their meaning is conditioned to the context to avoid ambiguity. In this work, we explore contextualized language models for NER in French biomedical text as part of the D{\'e}fi Fouille de Textes challenge. Our best approach achieved an F1 -measure of 66{\%} for symptoms and signs, and pathology categories, being top 1 for subtask 1. For anatomy, dose, exam, mode, moment, substance, treatment, and value categories, it achieved an F1 -measure of 75{\%} (subtask 2). If considered all categories, our model achieved the best result in the challenge, with an F1 -measure of 72{\%}. The use of an ensemble of neural language models proved to be very effective, improving a CRF baseline by up to 28{\%} and a single specialised language model by 4{\%}.
Approche supervis\'ee de calcul de similarit\'e s\'emantique entre paires de phrases (Supervised approach to compute semantic similarity between sentence pairs) (2020)

Ce papier d{\'e}crit les m{\'e}thodes que nous avons d{\'e}velopp{\'e}es pour participer aux t{\^a}ches 1 et 2 de l'{\'e}dition 2020 du d{\'e}fi fouille de textes (DEFT 2020). Pour la premi{\`e}re t{\^a}che, qui s`int{\'e}resse au calcul de scores de similarit{\'e} s{\'e}mantique entre paires de phrases, sur une {\'e}chelle de 0 {\`a} 5, une approche supervis{\'e}e o{\`u} chaque paire de phrases est repr{\'e}sent{\'e}e par un ensemble d`attributs a {\'e}t{\'e} propos{\'e}e. Des algorithmes classiques d`apprentissage automatique sont ensuite utilis{\'e}s pour entrainer les mod{\`e}les. Diff{\'e}rentes mesures de similarit{\'e} textuelle sont explor{\'e}es et les plus pertinentes sont combin{\'e}es pour supporter nos m{\'e}thodes. Diff{\'e}rentes combinaisons ont {\'e}t{\'e} test{\'e}es et {\'e}valu{\'e}es sur les donn{\'e}es de test du DEFT 2020. Notre meilleur syst{\`e}me qui s`appuie sur un mod{\`e}le Random Forest a obtenu les meilleures performances sur la premi{\`e}re t{\^a}che avec une EDRM de 0,8216.
DEFT 2020 - Extraction d`information fine dans les donn\'ees cliniques : terminologies sp\'ecialis\'ees et graphes de connaissance (Fine-grained Information Extraction in Clinical Data : Dedicated Terminologies and Knowledge Graphs ) (2020)

Nous pr{\'e}sentons dans cet article notre approche {\`a} base de r{\`e}gles con{\c{c}}ue pour r{\'e}pondre {\`a} la t{\^a}che 3 de la campagne d'{\'e}valuation DEFT 2020. Selon le type d`information {\`a} extraire, nous construisons (1) une terminologie sp{\'e}cialis{\'e}e {\`a} partir de ressources m{\'e}dicales et (2) un graphe orient{\'e} bas{\'e} sur les informations extraites de la base de connaissances g{\'e}n{\'e}raliste et de grande taille - JeuxDeMots.
DOING@DEFT : cascade de CRF pour l`annotation d`entit\'es cliniques imbriqu\'ees (DOING@DEFT : cascade of CRF for the annotation of nested clinical entities) (2020)

Cet article pr{\'e}sente le syst{\`e}me d{\'e}velopp{\'e} par l'{\'e}quipe DOING pour la campagne d'{\'e}valuation DEFT 2020 portant sur la similarit{\'e} s{\'e}mantique et l`extraction d`information fine. L'{\'e}quipe a particip{\'e} uniquement {\`a} la t{\^a}che 3 : {\textquotedblleft}extraction d`information{\textquotedblright}. Nous avons utilis{\'e} une cascade de CRF pour annoter les diff{\'e}rentes informations {\`a} rep{\'e}rer. Nous nous sommes concentr{\'e}s sur la question de l`imbrication des entit{\'e}s et de la pertinence d`un type d`entit{\'e} pour apprendre {\`a} reconna{\^i}tre un autre. Nous avons {\'e}galement test{\'e} l`utilisation d`une ressource externe, MedDRA, pour am{\'e}liorer les performances du syst{\`e}me et d`un pipeline plus complexe mais ne g{\'e}rant pas l`imbrication des entit{\'e}s. Nous avons soumis 3 runs et nous obtenons en moyenne sur toutes les classes des F-mesures de 0,64, 0,65 et 0,61.
Extraction d`information de sp\'ecialit\'e avec un syst\`eme commercial g\'en\'erique (Extracting Medical Information with an Off-the-shelf Software Product) (2020)

Nous avons particip{\'e} {\`a} la t{\^a}che 3 du D{\'e}fi Fouille de texte 2020, d{\'e}di{\'e}e {\`a} l`extraction d`information de sp{\'e}cialit{\'e}, dans le but de tester notre produit commercial d`extraction d`information, Watson Knowledge Studio (WKS), face {\`a} des {\'e}quipes acad{\'e}miques et industrielles. Outre la quantit{\'e} r{\'e}duite de donn{\'e}es d`apprentissage, la nature des annotations des corpus de r{\'e}f{\'e}rence posait des probl{\`e}mes d`adaptation {\`a} notre produit. Aussi avons-nous d{\^u} modifier le sch{\'e}ma d`annotation du corpus d`apprentissage, ex{\'e}cuter l`apprentissage, puis appliquer des r{\`e}gles aux r{\'e}sultats obtenus afin d`obtenir des annotations conformes au sch{\'e}ma initial. Nous avons {\'e}galement appliqu{\'e} des dictionnaires de sp{\'e}cialit{\'e} (anatomie, pathologie, etc.) pour injecter de la connaissance du domaine et renforcer les mod{\`e}les d`apprentissage automatique. Au final, nos r{\'e}sultats lors de la phase de test se situent dans la moyenne de l`ensemble des {\'e}quipes, avec des F-mesures de 0,43 pour la sous-t{\^a}che 1 et 0,63 pour la sous-t{\^a}che 2.
DEFT 2020 : d\'etection de similarit\'e entre phrases et extraction d`information (DEFT 2020 : sentence similarity detection and information retrieval ) (2020)

Ce papier d{\'e}crit la participation de Reezocar {\`a} la campagne d'{\'e}valuation DEFT 2020. Cette seizi{\`e}me {\'e}dition du challenge a port{\'e} sur le calcul de similarit{\'e} entre phrases et l`extraction d`information fine autour d`une douzaine de cat{\'e}gories dans des textes r{\'e}dig{\'e}s en Fran{\c{c}}ais. Le challenge propose trois t{\^a}ches : (i) la premi{\`e}re concerne l`identification du degr{\'e} de similarit{\'e} entre paires de phrases ; (ii) la deuxi{\`e}me concerne l`identification des phrases parall{\`e}les possibles pour une phrase source et (iii) la troisi{\`e}me concerne l`extraction d`information. Nous avons utilis{\'e} des m{\'e}thodes d`apprentissage automatique pour effectuer ces t{\^a}ches et avons obtenu des r{\'e}sultats satisfaisants sur l`ensemble des t{\^a}ches.
Similarit\'e s\'emantique entre phrases : apprentissage par transfert interlingue (Semantic Sentence Similarity : Multilingual Transfer Learning) (2020)

Dans cet article, nous d{\'e}crivons une approche exploratoire pour entra{\^i}ner des mod{\`e}les de langue et r{\'e}soudre des t{\^a}ches d`appariement entre phrases issues de corpus en fran{\c{c}}ais et relevant du domaine m{\'e}dical. Nous montrons que, dans un contexte o{\`u} les donn{\'e}es d`entra{\^i}nement sont en nombre restreint, il peut {\^e}tre int{\'e}ressant d`op{\'e}rer un apprentissage par transfert, d`une langue dont nous disposons de plus de ressources pour l`entra{\^i}nement, vers une langue cible moins dot{\'e}e de donn{\'e}es d`entra{\^i}nement (le fran{\c{c}}ais dans notre cas). Les r{\'e}sultats de nos exp{\'e}rimentations montrent que les mod{\`e}les de langue multilingues sont capables de transf{\'e}rer des repr{\'e}sentations d`une langue {\`a} l`autre de fa{\c{c}}on efficace pour r{\'e}soudre des t{\^a}ches de similarit{\'e} s{\'e}mantique telles que celles propos{\'e}es dans le cadre de l'{\'e}dition 2020 du D{\'e}fi fouille de texte (DEFT).
Participation de l'\'equipe du LIMICS \`a DEFT 2020 (Participation of team LIMICS in the DEFT 2020 challenge ) (2020)

Nous pr{\'e}sentons dans cet article les m{\'e}thodes con{\c{c}}ues et les r{\'e}sultats obtenus lors de notre participation {\`a} la t{\^a}che 3 de la campagne d'{\'e}valuation DEFT 2020, consistant en la reconnaissance d`entit{\'e}s nomm{\'e}es du domaine m{\'e}dical. Nous proposons deux mod{\`e}les diff{\'e}rents permettant de prendre en compte les entit{\'e}s imbriqu{\'e}es, qui repr{\'e}sentent une des difficult{\'e}s du jeu de donn{\'e}es propos{\'e}es, et pr{\'e}sentons les r{\'e}sultats obtenus. Notre meilleur run obtient la meilleure performance parmi les participants, sur l`une des deux sous-t{\^a}ches du d{\'e}fi.
Pratiques d'\'evaluation en ASR et biais de performance (Evaluation methodology in ASR and performance bias) (2020)

Nous proposons une r{\'e}flexion sur les pratiques d'{\'e}valuation des syst{\`e}mes de reconnaissance automatique de la parole (ASR). Apr{\`e}s avoir d{\'e}fini la notion de discrimination d`un point de vue l{\'e}gal et la notion d'{\'e}quit{\'e} dans les syst{\`e}mes d`intelligence artificielle, nous nous int{\'e}ressons aux pratiques actuelles lors des grandes campagnes d'{\'e}valuation. Nous observons que la variabilit{\'e} de la parole et plus particuli{\`e}rement celle de l`individu n`est pas prise en compte dans les protocoles d'{\'e}valuation actuels rendant impossible l'{\'e}tude de biais potentiels dans les syst{\`e}mes.
Comment arpenter sans m\`etre : les scores de r\'esolution de cha\^ines de cor\'ef\'erences sont-ils des m\'etriques ? (Do the standard scores of evaluation of coreference resolution constitute metrics ?) (2020)

Cet article pr{\'e}sente un travail qui consiste {\`a} {\'e}tudier si les scores les plus utilis{\'e}s pour l'{\'e}valuation de la r{\'e}solution des cor{\'e}f{\'e}rences constituent des m{\'e}triques de similarit{\'e} normalis{\'e}es. En adoptant une d{\'e}marche purement exp{\'e}rimentale, nous avons v{\'e}rifi{\'e} si les scores MUC, B3 , CEAF, BLANC, LEA et le meta-score CoNLL respectent les bonnes propri{\'e}t{\'e}s qui d{\'e}finissent une telle m{\'e}trique. Notre {\'e}tude montre que seul le score CEAFm est potentiellement une m{\'e}trique de similarit{\'e} normalis{\'e}e.
Que rec\`elent les donn\'ees textuelles issues du web ? (What do text data from the Web have to hide ?) (2020)

La collecte et l`usage opportunistes de donn{\'e}es textuelles tir{\'e}es du web sont sujets {\`a} une s{\'e}rie de probl{\`e}mes {\'e}thiques, m{\'e}thodologiques et {\'e}pist{\'e}mologiques qui m{\'e}ritent l`attention de la communaut{\'e} scientifique. Nous pr{\'e}sentons des {\'e}tudes empiriques de leur impact en linguistique et TAL centr{\'e}es sur la forme (m{\'e}thodes d`extraction des donn{\'e}es) ainsi que sur le fond (contenu des corpus).
R\'epliquer et \'etendre pour l`alsacien \textquotedblleft\'Etiquetage en parties du discours de langues peu dot\'ees par sp\'ecialisation des plongements lexicaux\textquotedblright (Replicating and extending for Alsatian : \textquotedblleftPOS tagging for low-resource languages by adapting word embeddings\textquotedblright) (2020)

Nous pr{\'e}sentons ici les r{\'e}sultats d`un travail de r{\'e}plication et d`extension pour l`alsacien d`une exp{\'e}rience concernant l'{\'e}tiquetage en parties du discours de langues peu dot{\'e}es par sp{\'e}cialisation des plongements lexicaux (Magistry et al., 2018). Ce travail a {\'e}t{\'e} r{\'e}alis{\'e} en {\'e}troite collaboration avec les auteurs de l`article d`origine. Cette interaction riche nous a permis de mettre au jour les {\'e}l{\'e}ments manquants dans la pr{\'e}sentation de l`exp{\'e}rience, de les compl{\'e}ter, et d'{\'e}tendre la recherche {\`a} la robustesse {\`a} la variation.
1990-2020 : retours sur 30 ans d'\'echanges autour de l`identification de voix en milieu judiciaire (1990-2020: A look back at 30 years of discussions on voice identification in the judicial system) (2020)

Des enregistrements de voix se trouvent de plus en plus souvent au c{\oe}ur d`affaires judiciaires importantes, notamment de par l`essor de la t{\'e}l{\'e}phonie mobile. La justice demande {\`a} ce que des expertises en identification de voix soient r{\'e}alis{\'e}es alors que dans le m{\^e}me temps, la pertinence scientifique de telles expertises est fortement mise en cause par les scientifiques. Ainsi, d{\`e}s 1990, les chercheurs en communication parl{\'e}e r{\'e}unis dans le GFCP, devenu depuis AFCP, ont vot{\'e} une motion affirmant que {\guillemotleft} l`identification d`un individu par sa voix est {\`a} l`heure actuelle un probl{\`e}me {\`a} sa connaissance non r{\'e}solu {\guillemotright}. Cette motion est toujours en vigueur, apr{\`e}s avoir {\'e}t{\'e} r{\'e}affirm{\'e}e en 1997 et renforc{\'e}e par une p{\'e}tition en 2002. Malgr{\'e} cela, des expertises judiciaires en identification de voix sont r{\'e}alis{\'e}es en France chaque ann{\'e}e. Cet article revient sur les actions men{\'e}es par le GFCP et l`AFCP depuis la motion initiale jusqu`aux actions contemporaines. Il se propose d'{\'e}valuer les r{\'e}percussions de ces actions, tant au niveau de la Justice qu`au niveau acad{\'e}mique.
FINDINGS OF THE IWSLT 2020 EVALUATION CAMPAIGN (2020)

The evaluation campaign of the International Conference on Spoken Language Translation (IWSLT 2020) featured this year six challenge tracks: (i) Simultaneous speech translation, (ii) Video speech translation, (iii) Offline speech translation, (iv) Conversational speech translation, (v) Open domain translation, and (vi) Non-native speech translation. A total of teams participated in at least one of the tracks. This paper introduces each track`s goal, data and evaluation metrics, and reports the results of the received submissions.
ON-TRAC Consortium for End-to-End and Simultaneous Speech Translation Challenge Tasks at IWSLT 2020 (2020)

This paper describes the ON-TRAC Consortium translation systems developed for two challenge tracks featured in the Evaluation Campaign of IWSLT 2020, offline speech translation and simultaneous speech translation. ON-TRAC Consortium is composed of researchers from three French academic laboratories: LIA (Avignon Universit{\'e}), LIG (Universit{\'e} Grenoble Alpes), and LIUM (Le Mans Universit{\'e}). Attention-based encoder-decoder models, trained end-to-end, were used for our submissions to the offline speech translation track. Our contributions focused on data augmentation and ensembling of multiple models. In the simultaneous speech translation track, we build on Transformer-based wait-k models for the text-to-text subtask. For speech-to-text simultaneous translation, we attach a wait-k MT system to a hybrid ASR system. We propose an algorithm to control the latency of the ASR+MT cascade and achieve a good latency-quality trade-off on both subtasks.
Start-Before-End and End-to-End: Neural Speech Translation by AppTek and RWTH Aachen University (2020)

AppTek and RWTH Aachen University team together to participate in the offline and simultaneous speech translation tracks of IWSLT 2020. For the offline task, we create both cascaded and end-to-end speech translation systems, paying attention to careful data selection and weighting. In the cascaded approach, we combine high-quality hybrid automatic speech recognition (ASR) with the Transformer-based neural machine translation (NMT). Our end-to-end direct speech translation systems benefit from pretraining of adapted encoder and decoder components, as well as synthetic data and fine-tuning and thus are able to compete with cascaded systems in terms of MT quality. For simultaneous translation, we utilize a novel architecture that makes dynamic decisions, learned from parallel data, to determine when to continue feeding on input or generate output words. Experiments with speech and text input show that even at low latency this architecture leads to superior translation results.
KIT`s IWSLT 2020 SLT Translation System (2020)

This paper describes KIT`s submissions to the IWSLT2020 Speech Translation evaluation campaign. We first participate in the simultaneous translation task, in which our simultaneous models are Transformer based and can be efficiently trained to obtain low latency with minimized compromise in quality. On the offline speech translation task, we applied our new Speech Transformer architecture to end-to-end speech translation. The obtained model can provide translation quality which is competitive to a complicated cascade. The latter still has the upper hand, thanks to the ability to transparently access to the transcription, and resegment the inputs to avoid fragmentation.
End-to-End Simultaneous Translation System for IWSLT2020 Using Modality Agnostic Meta-Learning (2020)

In this paper, we describe end-to-end simultaneous speech-to-text and text-to-text translation systems submitted to IWSLT2020 online translation challenge. The systems are built by adding wait-k and meta-learning approaches to the Transformer architecture. The systems are evaluated on different latency regimes. The simultaneous text-to-text translation achieved a BLEU score of 26.38 compared to the competition baseline score of 14.17 on the low latency regime (Average latency {\ensuremath{\leq}} 3). The simultaneous speech-to-text system improves the BLEU score by 7.7 points over the competition baseline for the low latency regime (Average Latency {\ensuremath{\leq}} 1000).
DiDi Labs' End-to-end System for the IWSLT 2020 Offline Speech TranslationTask (2020)

This paper describes the system that was submitted by DiDi Labs to the offline speech translation task for IWSLT 2020. We trained an end-to-end system that translates audio from English TED talks to German text, without producing intermediate English text. We use the S-Transformer architecture and train using the MuSTC dataset. We also describe several additional experiments that were attempted, but did not yield improved results.
End-to-End Offline Speech Translation System for IWSLT 2020 using Modality Agnostic Meta-Learning (2020)

In this paper, we describe the system submitted to the IWSLT 2020 Offline Speech Translation Task. We adopt the Transformer architecture coupled with the meta-learning approach to build our end-to-end Speech-to-Text Translation (ST) system. Our meta-learning approach tackles the data scarcity of the ST task by leveraging the data available from Automatic Speech Recognition (ASR) and Machine Translation (MT) tasks. The meta-learning approach combined with synthetic data augmentation techniques improves the model performance significantly and achieves BLEU scores of 24.58, 27.51, and 27.61 on IWSLT test 2015, MuST-C test, and Europarl-ST test sets respectively.
End-to-End Speech-Translation with Knowledge Distillation: FBK@IWSLT2020 (2020)

This paper describes FBK`s participation in the IWSLT 2020 offline speech translation (ST) task. The task evaluates systems' ability to translate English TED talks audio into German texts. The test talks are provided in two versions: one contains the data already segmented with automatic tools and the other is the raw data without any segmentation. Participants can decide whether to work on custom segmentation or not. We used the provided segmentation. Our system is an end-to-end model based on an adaptation of the Transformer for speech data. Its training process is the main focus of this paper and it is based on: i) transfer learning (ASR pretraining and knowledge distillation), ii) data augmentation (SpecAugment, time stretch and synthetic data), iii)combining synthetic and real data marked as different domains, and iv) multi-task learning using the CTC loss. Finally, after the training with word-level knowledge distillation is complete, our ST models are fine-tuned using label smoothed cross entropy. Our best model scored 29 BLEU on the MuST-CEn-De test set, which is an excellent result compared to recent papers, and 23.7 BLEU on the same data segmented with VAD, showing the need for researching solutions addressing this specific data condition.
SRPOL`s System for the IWSLT 2020 End-to-End Speech Translation Task (2020)

We took part in the offline End-to-End English to German TED lectures translation task. We based our solution on our last year`s submission. We used a slightly altered Transformer architecture with ResNet-like convolutional layer preparing the audio input to Transformer encoder. To improve the model`s quality of translation we introduced two regularization techniques and trained on machine translated Librispeech corpus in addition to iwslt-corpus, TEDLIUM2 andMust{\_}C corpora. Our best model scored almost 3 BLEU higher than last year`s model. To segment 2020 test set we used exactly the same procedure as last year.
The University of Helsinki Submission to the IWSLT2020 Offline SpeechTranslation Task (2020)

This paper describes the University of Helsinki Language Technology group`s participation in the IWSLT 2020 offline speech translation task, addressing the translation of English audio into German text. In line with this year`s task objective, we train both cascade and end-to-end systems for spoken language translation. We opt for an end-to-end multitasking architecture with shared internal representations and a cascade approach that follows a standard procedure consisting of ASR, correction, and MT stages. We also describe the experiments that served as a basis for the submitted systems. Our experiments reveal that multitasking training with shared internal representations is not only possible but allows for knowledge-transfer across modalities.
The AFRL IWSLT 2020 Systems: Work-From-Home Edition (2020)

This report summarizes the Air Force Research Laboratory (AFRL) submission to the offline spoken language translation (SLT) task as part of the IWSLT 2020 evaluation campaign. As in previous years, we chose to adopt the cascade approach of using separate systems to perform speech activity detection, automatic speech recognition, sentence segmentation, and machine translation. All systems were neural based, including a fully-connected neural network for speech activity detection, a Kaldi factorized time delay neural network with recurrent neural network (RNN) language model rescoring for speech recognition, a bidirectional RNN with attention mechanism for sentence segmentation, and transformer networks trained with OpenNMT and Marian for machine translation. Our primary submission yielded BLEU scores of 21.28 on tst2019 and 23.33 on tst2020.
LIT Team`s System Description for Japanese-Chinese Machine Translation Task in IWSLT 2020 (2020)

This paper describes the LIT Team`s submission to the IWSLT2020 open domain translation task, focusing primarily on Japanese-to-Chinese translation direction. Our system is based on the organizers' baseline system, but we do more works on improving the Transform baseline system by elaborate data pre-processing. We manage to obtain significant improvements, and this paper aims to share some data processing experiences in this translation task. Large-scale back-translation on monolingual corpus is also investigated. In addition, we also try shared and exclusive word embeddings, compare different granularity of tokens like sub-word level. Our Japanese-to-Chinese translation system achieves a performance of BLEU=34.0 and ranks 2nd among all participating systems.
OPPO`s Machine Translation System for the IWSLT 2020 Open Domain Translation Task (2020)

In this paper, we demonstrate our machine translation system applied for the Chinese-Japanese bidirectional translation task (aka. open domain translation task) for the IWSLT 2020. Our model is based on Transformer (Vaswani et al., 2017), with the help of many popular, widely proved effective data preprocessing and augmentation methods. Experiments show that these methods can improve the baseline model steadily and significantly.
Character Mapping and Ad-hoc Adaptation: Edinburgh`s IWSLT 2020 Open Domain Translation System (2020)

This paper describes the University of Edinburgh`s neural machine translation systems submitted to the IWSLT 2020 open domain Japanese$\leftrightarrow$Chinese translation task. On top of commonplace techniques like tokenisation and corpus cleaning, we explore character mapping and unsupervised decoding-time adaptation. Our techniques focus on leveraging the provided data, and we show the positive impact of each technique through the gradual improvement of BLEU.