ymoslem/acl-abstracts huggingface dataset:
Reconnaissance d`entit\'es nomm\'ees sur des sorties OCR bruit\'ees : des pistes pour la d\'esambigu\"isation morphologique automatique (Resolution of entity linking issues on noisy OCR output : automatic disambiguation tracks) (2022)

La variation dans les donn{\'e}es textuelles, en particulier le bruit, est un facteur limitant la performance des syst{\`e}mes de Reconnaissance d`Entit{\'e}s Nomm{\'e}es (REN). Les syst{\`e}mes de REN sont en effet g{\'e}n{\'e}ralement entra{\^i}n{\'e}s sur des donn{\'e}es {\guillemotleft} propres {\guillemotright}, non-bruit{\'e}es, ce qui n`est pas le cas des donn{\'e}es des humanit{\'e}s num{\'e}riques obtenues par reconnaissance optique de caract{\`e}res (OCR). De fait, la qualit{\'e} des transcriptions OCR est souvent per{\c{c}}ue comme la source principale des erreurs faites par les outils de REN. Cependant, des r{\'e}sultats obtenus avec diff{\'e}rents syst{\`e}mes REN sur des transcriptions OCR d`un corpus du 19{\`e}me si{\`e}cle (ELTeC) tendent {\`a} montrer une certaine robustesse, modulo la pr{\'e}sence de formes bruit{\'e}es, parfois dites {\guillemotleft} contamin{\'e}es {\guillemotright}. La difficult{\'e}, est alors de lier ces formes contamin{\'e}es avec leur forme de r{\'e}f{\'e}rence, par exemple, pour rapprocher la cha{\^i}ne {\guillemotleft} Parisl {\guillemotright}et la cha{\^i}ne {\guillemotleft} Paris {\guillemotright}. Il s`agit de mod{\'e}liser le fait que diff{\'e}rentes variations se rapprochent du m{\^e}me terme. Des questions quant {\`a} l`automatisation de cette t{\^a}che et sa g{\'e}n{\'e}ralisation {\`a} toutes les variations d`un m{\^e}me terme restent ouvertes. Nous montrons dans cet article diff{\'e}rentes exp{\'e}riences visant {\`a} traiter ce probl{\`e}me sous l{\textquoteleft}angle de la d{\'e}sambigu{\"i}sation morphologique des entit{\'e}s nomm{\'e}es (EN) en aval de la cha{\^i}ne de traitement, plut{\^o}t que par la correction en amont des donn{\'e}es de l`OCR.
R\'einterroger l'\'edition num\'erique et la consultation d`oeuvres anciennes : tra\ccabilit\'e, accessibilit\'e, interpr\'etabilit\'e (2022)

Dans le domaine des humanit{\'e}s num{\'e}riques et de l'{\'e}dition d`oeuvres anciennes, l`influence de la Text Encoding Initiative (TEI) a port{\'e} ses fruits et n`est plus {\`a} d{\'e}montrer. Le contexte technologique est cependant propice {\`a} l'{\'e}mergence de nouveaux modes de consultation et de diffusion. Nous nous appuierons sur la cr{\'e}ation d`une nouvelle interface de consultation des oeuvres de Descartes pour traiter des questions de tra{\c{c}}abilit{\'e} des op{\'e}rations, d`interop{\'e}rabilit{\'e} des ressources de TAL, et d`interpr{\'e}tabilit{\'e}.
Romanciers et romanci\`eres du XIX\`eme si\`ecle : une \'etude automatique du genre sur le corpus GIRLS (Male and female novelists : an automatic study of gender of authors and their characters ) (2022)

Cette {\'e}tude porte sur les diff{\'e}rences entre les romans fran{\c{c}}ais du XIXe si{\`e}cle {\'e}crits par des hommes et ceux {\'e}crits par des femmes en trois {\'e}tapes. Premi{\`e}rement, nous observons que ces textes peuvent {\^e}tre distingu{\'e}s par apprentissage supervis{\'e} selon ce crit{\`e}re. Un mod{\`e}le simple a un score de 99{\%} d`exactitude sur cette t{\^a}che si d`autres {\oe}uvres de la m{\^e}me personne figurent dans le jeu d`entra{\^i}nement, et de 72{\%} d`exactitude sinon. Cette diff{\'e}rence s`explique par le fait que le langage de l`individu est plus distinctif qu`un {\'e}ventuel style propre au genre. Deuxi{\`e}mement, notre {\'e}tude textom{\'e}trique met au jour des st{\'e}r{\'e}otypes de genre chez les hommes et les femmes. Troisi{\`e}mement, nous pr{\'e}sentons un mod{\`e}le de cor{\'e}f{\'e}rence entra{\^i}n{\'e} sur des textes litt{\'e}raires pour {\'e}tudier le genre des personnages. Nous montrons ainsi que les personnages f{\'e}minins sont plus nombreux chez les femmes, et prennent g{\'e}n{\'e}ralement une place plus pro{\'e}minente que chez les hommes.
Simulation d`erreurs d'OCR dans les syst\`emes de TAL pour le traitement de donn\'ees anachroniques (Simulation of OCR errors in NLP systems for processing anachronistic data) (2022)

L`extraction d`information offre de nouvelles perspectives au sein des recherches historiques. Cependant, la majorit{\'e} des recherches li{\'e}es {\`a} ce domaine s`effectue sur des donn{\'e}es contemporaines. Malgr{\'e} l'{\'e}volution constante des syst{\`e}mes d`OCR, les textes historiques r{\'e}sultant de ce proc{\'e}d{\'e} contiennent toujours de multiples erreurs. Du fait d`un manque de ressources historiques d{\'e}di{\'e}es au TAL, le traitement de ce domaine reste d{\'e}pendant de l`utilisation de ressources contemporaines. De nombreuses {\'e}tudes ont d{\'e}montr{\'e} l`impact n{\'e}gatif que pouvaient avoir les erreurs d`OCR sur les syst{\`e}mes pr{\^e}ts {\`a} l`emploi contemporains. Mais l'{\'e}valuation des nouvelles architectures, proposant des r{\'e}sultats prometteurs sur des donn{\'e}es r{\'e}centes, face {\`a} ce probl{\`e}me reste encore tr{\`e}s minime. Dans cette {\'e}tude, nous quantifions l`impact des erreurs d`OCR sur trois t{\^a}ches d`extraction d`information en utilisant plusieurs architectures de type Transformers. Au vu de ces r{\'e}sultats, nous proposons une approche permettant de r{\'e}duire de plus de 50{\%} cet impact sans avoir recours {\`a} des ressources historiques sp{\'e}cialis{\'e}es.
TAL et Litt\'erature compar\'ee. D\'etection automatique des correspondances textuelles entre les r\'e\'ecritures d`un mythe (NLP and Comparative Literature) (2022)

L`id{\'e}e de pouvoir d{\'e}tecter automatiquement des relations intertextuelles est stimulante, pour la recherche litt{\'e}raire et linguistique, et pour l'{\'e}dition num{\'e}rique. Cependant, si les logiciels employ{\'e}s pour notre projet, TextPAIR et Tracer, sont tr{\`e}s performants pour les correspondances proches, gr{\^a}ce {\`a} des techniques de l`intelligence artificielle, ils ne d{\'e}tectent pas (bien) des r{\'e}utilisations et {\'e}vocations plus complexes. Nous proposons d`am{\'e}liorer les r{\'e}sultats en faisant coop{\'e}rer l`herm{\'e}neutique sp{\'e}cifique des {\'e}tudes litt{\'e}raires avec des m{\'e}thodes talistes, linguistiques et informatiques. Nous rencontrons toutefois quelques difficult{\'e}s en traitant notre corpus avec des outils du TAL.
SubER - A Metric for Automatic Evaluation of Subtitle Quality (2022)

This paper addresses the problem of evaluating the quality of automatically generated subtitles, which includes not only the quality of the machine-transcribed or translated speech, but also the quality of line segmentation and subtitle timing. We propose SubER - a single novel metric based on edit distance with shifts that takes all of these subtitle properties into account. We compare it to existing metrics for evaluating transcription, translation, and subtitle quality. A careful human evaluation in a post-editing scenario shows that the new metric has a high correlation with the post-editing effort and direct human assessment scores, outperforming baseline metrics considering only the subtitle text, such as WER and BLEU, and existing methods to integrate segmentation and timing features.
Improving Arabic Diacritization by Learning to Diacritize and Translate (2022)

We propose a novel multitask learning method for diacritization which trains a model to both diacritize and translate. Our method addresses data sparsity by exploiting large, readily available bitext corpora. Furthermore, translation requires implicit linguistic and semantic knowledge, which is helpful for resolving ambiguities in diacritization. We apply our method to the Penn Arabic Treebank and report a new state-of-the-art word error rate of 4.79{\%}. We also conduct manual and automatic analysis to better understand our method and highlight some of the remaining challenges in diacritization. Our method has applications in text-to-speech, speech-to-speech translation, and other NLP tasks.
Simultaneous Neural Machine Translation with Prefix Alignment (2022)

Simultaneous translation is a task that requires starting translation before the speaker has finished speaking, so we face a trade-off between latency and accuracy. In this work, we focus on prefix-to-prefix translation and propose a method to extract alignment between bilingual prefix pairs. We use the alignment to segment a streaming input and fine-tune a translation model. The proposed method demonstrated higher BLEU than those of baselines in low latency ranges in our experiments on the IWSLT simultaneous translation benchmark.
Locality-Sensitive Hashing for Long Context Neural Machine Translation (2022)

After its introduction the Transformer architecture quickly became the gold standard for the task of neural machine translation. A major advantage of the Transformer compared to previous architectures is the faster training speed achieved by complete parallelization across timesteps due to the use of attention over recurrent layers. However, this also leads to one of the biggest problems of the Transformer, namely the quadratic time and memory complexity with respect to the input length. In this work we adapt the locality-sensitive hashing approach of Kitaev et al. (2020) to self-attention in the Transformer, we extended it to cross-attention and apply this memory efficient framework to sentence- and document-level machine translation. Our experiments show that the LSH attention scheme for sentence-level comes at the cost of slightly reduced translation quality. For document-level NMT we are able to include much bigger context sizes than what is possible with the baseline Transformer. However, more context does neither improve translation quality nor improve scores on targeted test suites.
Anticipation-Free Training for Simultaneous Machine Translation (2022)

Simultaneous machine translation (SimulMT) speeds up the translation process by starting to translate before the source sentence is completely available. It is difficult due to limited context and word order difference between languages. Existing methods increase latency or introduce adaptive read-write policies for SimulMT models to handle local reordering and improve translation quality. However, the long-distance reordering would make the SimulMT models learn translation mistakenly. Specifically, the model may be forced to predict target tokens when the corresponding source tokens have not been read. This leads to aggressive anticipation during inference, resulting in the hallucination phenomenon. To mitigate this problem, we propose a new framework that decompose the translation process into the monotonic translation step and the reordering step, and we model the latter by the auxiliary sorting network (ASN). The ASN rearranges the hidden states to match the order in the target language, so that the SimulMT model could learn to translate more reasonably. The entire model is optimized end-to-end and does not rely on external aligners or data. During inference, ASN is removed to achieve streaming. Experiments show the proposed framework could outperform previous methods with less latency.
Who Are We Talking About? Handling Person Names in Speech Translation (2022)

Recent work has shown that systems for speech translation (ST) {--} similarly to automatic speech recognition (ASR) {--} poorly handle person names. This shortcoming does not only lead to errors that can seriously distort the meaning of the input, but also hinders the adoption of such systems in application scenarios (like computer-assisted interpreting) where the translation of named entities, like person names, is crucial. In this paper, we first analyse the outputs of ASR/ST systems to identify the reasons of failures in person name transcription/translation. Besides the frequency in the training data, we pinpoint the nationality of the referred person as a key factor. We then mitigate the problem by creating multilingual models, and further improve our ST systems by forcing them to jointly generate transcripts and translations, prioritising the former over the latter. Overall, our solutions result in a relative improvement in token-level person name accuracy by 47.8{\%} on average for three language pairs (en-{\ensuremath{>}}es,fr,it).
Joint Generation of Captions and Subtitles with Dual Decoding (2022)

As the amount of audio-visual content increases, the need to develop automatic captioning and subtitling solutions to match the expectations of a growing international audience appears as the only viable way to boost throughput and lower the related post-production costs. Automatic captioning and subtitling often need to be tightly intertwined to achieve an appropriate level of consistency and synchronization with each other and with the video signal. In this work, we assess a dual decoding scheme to achieve a strong coupling between these two tasks and show how adequacy and consistency are increased, with virtually no additional cost in terms of model size and training complexity.
MirrorAlign: A Super Lightweight Unsupervised Word Alignment Model via Cross-Lingual Contrastive Learning (2022)

Word alignment is essential for the downstream cross-lingual language understanding and generation tasks. Recently, the performance of the neural word alignment models has exceeded that of statistical models. However, they heavily rely on sophisticated translation models. In this study, we propose a super lightweight unsupervised word alignment model named MirrorAlign, in which bidirectional symmetric attention trained with a contrastive learning objective is introduced, and an agreement loss is employed to bind the attention maps, such that the alignments follow mirror-like symmetry hypothesis. Experimental results on several public benchmarks demonstrate that our model achieves competitive, if not better, performance compared to the state of the art in word alignment while significantly reducing the training and decoding time on average. Further ablation analysis and case studies show the superiority of our proposed MirrorAlign. Notably, we recognize our model as a pioneer attempt to unify bilingual word embedding and word alignments. Encouragingly, our approach achieves 16.4X speedup against GIZA++, and 50X parameter compression compared with the Transformer-based alignment methods. We release our code to facilitate the community: \url{https://github.com/moore3930/MirrorAlign}.
On the Impact of Noises in Crowd-Sourced Data for Speech Translation (2022)

Training speech translation (ST) models requires large and high-quality datasets. MuST-C is one of the most widely used ST benchmark datasets. It contains around 400 hours of speech-transcript-translation data for each of the eight translation directions. This dataset passes several quality-control filters during creation. However, we find that MuST-C still suffers from three major quality issues: audiotext misalignment, inaccurate translation, and unnecessary speaker`s name. What are the impacts of these data quality issues for model development and evaluation? In this paper, we propose an automatic method to fix or filter the above quality issues, using English-German (En-De) translation as an example. Our experiments show that ST models perform better on clean test sets, and the rank of proposed models remains consistent across different test sets. Besides, simply removing misaligned data points from the training set does not lead to a better ST model.
Findings of the IWSLT 2022 Evaluation Campaign (2022)

The evaluation campaign of the 19th International Conference on Spoken Language Translation featured eight shared tasks: (i) Simultaneous speech translation, (ii) Offline speech translation, (iii) Speech to speech translation, (iv) Low-resource speech translation, (v) Multilingual speech translation, (vi) Dialect speech translation, (vii) Formality control for speech translation, (viii) Isometric speech translation. A total of 27 teams participated in at least one of the shared tasks. This paper details, for each shared task, the purpose of the task, the data that were released, the evaluation metrics that were applied, the submissions that were received and the results that were achieved.
The YiTrans Speech Translation System for IWSLT 2022 Offline Shared Task (2022)

This paper describes the submission of our end-to-end YiTrans speech translation system for the IWSLT 2022 offline task, which translates from English audio to German, Chinese, and Japanese. The YiTrans system is built on large-scale pre-trained encoder-decoder models. More specifically, we first design a multi-stage pre-training strategy to build a multi-modality model with a large amount of labeled and unlabeled data. We then fine-tune the corresponding components of the model for the downstream speech translation tasks. Moreover, we make various efforts to improve performance, such as data filtering, data augmentation, speech segmentation, model ensemble, and so on. Experimental results show that our YiTrans system obtains a significant improvement than the strong baseline on three translation directions, and it achieves +5.2 BLEU improvements over last year`s optimal end-to-end system on tst2021 English-German.
Amazon Alexa AI`s System for IWSLT 2022 Offline Speech Translation Shared Task (2022)

This paper describes Amazon Alexa AI`s submission to the IWSLT 2022 Offline Speech Translation Task. Our system is an end-to-end speech translation model that leverages pretrained models and cross modality transfer learning. We detail two improvements to the knowledge transfer schema. First, we implemented a new loss function that reduces knowledge gap between audio and text modalities in translation task effectively. Second, we investigate multiple finetuning strategies including sampling loss, language grouping and domain adaption. These strategies aims to bridge the gaps between speech and text translation tasks. We also implement a multi-stage segmentation and merging strategy that yields improvements on the unsegmented development datasets. Results show that the proposed loss function consistently improves BLEU scores on the development datasets for both English-German and multilingual models. Additionally, certain language pairs see BLEU score improvements with specific finetuning strategies.
Efficient yet Competitive Speech Translation: FBK@IWSLT2022 (2022)

The primary goal of this FBK`s systems submission to the IWSLT 2022 offline and simultaneous speech translation tasks is to reduce model training costs without sacrificing translation quality. As such, we first question the need of ASR pre-training, showing that it is not essential to achieve competitive results. Second, we focus on data filtering, showing that a simple method that looks at the ratio between source and target characters yields a quality improvement of 1 BLEU. Third, we compare different methods to reduce the detrimental effect of the audio segmentation mismatch between training data manually segmented at sentence level and inference data that is automatically segmented. Towards the same goal of training cost reduction, we participate in the simultaneous task with the same model trained for offline ST. The effectiveness of our lightweight training strategy is shown by the high score obtained on the MuST-C en-de corpus (26.7 BLEU) and is confirmed in high-resource data conditions by a 1.6 BLEU improvement on the IWSLT2020 test set over last year`s winning system.
Effective combination of pretrained models - KIT@IWSLT2022 (2022)

Pretrained models in acoustic and textual modalities can potentially improve speech translation for both Cascade and End-to-end approaches. In this evaluation, we aim at empirically looking for the answer by using the wav2vec, mBART50 and DeltaLM models to improve text and speech translation models. The experiments showed that the presence of these models together with an advanced audio segmentation method results in an improvement over the previous end-to-end system by up to 7 BLEU points. More importantly, the experiments showed that given enough data and modeling capacity to overcome the training difficulty, we can outperform even very competitive Cascade systems. In our experiments, this gap can be as large as 2.0 BLEU points, the same gap that the Cascade often led over the years.
The USTC-NELSLIP Offline Speech Translation Systems for IWSLT 2022 (2022)

This paper describes USTC-NELSLIP`s submissions to the IWSLT 2022 Offline Speech Translation task, including speech translation of talks from English to German, English to Chinese and English to Japanese. We describe both cascaded architectures and end-to-end models which can directly translate source speech into target text. In the cascaded condition, we investigate the effectiveness of different model architectures with robust training and achieve 2.72 BLEU improvements over last year`s optimal system on MuST-C English-German test set. In the end-to-end condition, we build models based on Transformer and Conformer architectures, achieving 2.26 BLEU improvements over last year`s optimal end-to-end system. The end-to-end system has obtained promising results, but it is still lagging behind our cascaded models.
The AISP-SJTU Simultaneous Translation System for IWSLT 2022 (2022)

This paper describes AISP-SJTU`s submissions for the IWSLT 2022 Simultaneous Translation task. We participate in the text-to-text and speech-to-text simultaneous translation from English to Mandarin Chinese. The training of the CAAT is improved by training across multiple values of right context window size, which achieves good online performance without setting a prior right context window size for training. For speech-to-text task, the best model we submitted achieves 25.87, 26.21, 26.45 BLEU in low, medium and high regimes on tst-COMMON, corresponding to 27.94, 28.31, 28.43 BLEU in text-to-text task.
The Xiaomi Text-to-Text Simultaneous Speech Translation System for IWSLT 2022 (2022)

This system paper describes the Xiaomi Translation System for the IWSLT 2022 Simultaneous Speech Translation (noted as SST) shared task. We participate in the English-to-Mandarin Chinese Text-to-Text (noted as T2T) track. Our system is built based on the Transformer model with novel techniques borrowed from our recent research work. For the data filtering, language-model-based and rule-based methods are conducted to filter the data to obtain high-quality bilingual parallel corpora. We also strengthen our system with some dominating techniques related to data augmentation, such as knowledge distillation, tagged back-translation, and iterative back-translation. We also incorporate novel training techniques such as R-drop, deep model, and large batch training which have been shown to be beneficial to the naive Transformer model. In the SST scenario, several variations of $exttt{wait-k}$ strategies are explored. Furthermore, in terms of robustness, both data-based and model-based ways are used to reduce the sensitivity of our system to Automatic Speech Recognition (ASR) outputs. We finally design some inference algorithms and use the adaptive-ensemble method based on multiple model variants to further improve the performance of the system. Compared with strong baselines, fusing all techniques can improve our system by 2 extasciitilde3 BLEU scores under different latency regimes.
NVIDIA NeMo Offline Speech Translation Systems for IWSLT 2022 (2022)

This paper provides an overview of NVIDIA NeMo`s speech translation systems for the IWSLT 2022 Offline Speech Translation Task. Our cascade system consists of 1) Conformer RNN-T automatic speech recognition model, 2) punctuation-capitalization model based on pre-trained T5 encoder, 3) ensemble of Transformer neural machine translation models fine-tuned on TED talks. Our end-to-end model has less parameters and consists of Conformer encoder and Transformer decoder. It relies on the cascade system by re-using its pre-trained ASR encoder and training on synthetic translations generated with the ensemble of NMT models. Our En-{\ensuremath{>}}De cascade and end-to-end systems achieve 29.7 and 26.2 BLEU on the 2020 test set correspondingly, both outperforming the previous year`s best of 26 BLEU.