Original: L ET Y OUR H EART S PEAK IN ITS M OTHER T ONGUE : M ULTILINGUAL C APTIONING OF C ARDIAC S IGNALS
Revised: L ET Y OUR H EART S PEAK IN ITS M OTHER T ONGUE : M ULTILINGUAL C APTIONING OF C ARDIAC S IGNALS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Cardiac signals convey a signiﬁcant amount of information about the health status of a patient.
Revised: Cardiac signals convey a signiﬁcant amount of information about the health status of a patient.

Original: Upon recording these signals, cardiologists are expected to manually generate an accompanying report to share with physicians and patients.
Revised: Upon recording these signals, cardiologists are expected to manually generate an accompanying report to share with physicians and patients.

Original: Generating these reports, however, can be time-consuming and error-prone, while also exhibiting a high degree of intra- and inter-physician variability.
Revised: Generating these reports, however, can be time-consuming and error-prone, while also exhibiting a high degree of intra- and inter-physician variability.

Original: To address this, we design a neural, multilingual, cardiac signal captioning framework.
Revised: To address this, we design a neural, multilingual, cardiac signal captioning framework.

Original: In the process, we propose a discriminative multilingual representation learning method, RTLP, which randomly replaces tokens with those from a different language and tasks a network with identifying the language of all tokens.
Revised: In the process, we propose a discriminative multilingual representation learning method, RTLP, which randomly replaces tokens with those from a different language and tasks a network with identifying the language of all tokens.

Original: We show that RTLP performs on par with  state-of-the-artpre-training methods such as MLM and MARGE, while generating more clinically accurate reports than MLM.
Revised: We show that RTLPgenerated reports are of high quality and clinical utility, and are on par with reports generated by networks pre-trained with state-of-the-art methods such as MLM and MARGE.

Original: We also show that, with RTLP, multilingual ﬁne-tuning can be preferable to its monolingual counterpart, a phenomenon we refer to as the blessing of multilinguality .
Revised: We also show that generated reports exhibit higher quality and clinical utility when RTLP is ﬁne-tuned in a multilingual setting than in a monolingual setting, a phenomenon we refer to as the blessing of multilinguality .

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Cardiac signals convey a signiﬁcant amount of information about the status of a patient.
Revised: Cardiac signals convey a signiﬁcant amount of information about the status of a patient.

Original: Upon recording these signals, an expert cardiologist is often required to interpret the ﬁndings, manually generate an accompanying textual report (Richley & Walters, 2020), and share it with fellow physicians (and patients).
Revised: Upon recording these signals, an expert cardiologist is often required to interpret the ﬁndings, manually generate an accompanying textual report (Richley & Walters, 2020), and share it with fellow physicians (and patients).

Original: Such a report is crucial within medicine, impacting communication between physicians, facilitating clinical decision-making, and holding care teams accountable for their actions (Waegemann et al., 2002).
Revised: Such a report is crucial within medicine, impacting communication between physicians, facilitating clinical decision-making, and holding care teams accountable for their actions (Waegemann et al., 2002).

Original: Manually generating these reports, however, can be time-consuming and error-prone, while also exhibiting a high degree of intra and inter-physician variability.
Revised: Manually generating these reports, however, can be time-consuming and error-prone, while also exhibiting a high degree of intra and inter-physician variability.

Original: Combined, these can detract from a physician’s time with a patient, thus decreasing patient satisfaction, and hinder communication between physicians (Hibbard et al., 2001; Keselman & Smith, 2012), thus potentially compromising patient outcomes (Brailer et al., 1997).
Revised: Combined, these can detract from a physician’s time with a patient, thus decreasing patient satisfaction, and hinder communication between physicians (Hibbard et al., 2001; Keselman & Smith, 2012), thus potentially compromising patient outcomes (Brailer et al., 1997).

Revised: One way to address this challenge is by collapsing a clinical report into multiple pertinent medical conditions and designing a system that identiﬁes such conditions.

Revised: However, abstracting away the details of a report can obscure the full assessment of a patient’s health status and deﬁning the pertinence of conditions can be disease-speciﬁc and non-trivial.

Original: Based on these observations, we look to address the following question: how do we automatically generate clinically accurate reports that reliably summarize cardiac signals?
Revised: Based on these observations, we look to address the following question: how do we automatically generate clinically accurate reports that reliably summarize cardiac signals?

Original: To address this question, the ﬁelds of visual-language representation learning and captioning hold promise.
Revised: To address this question, the ﬁelds of visual-language representation learning and captioning hold promise.

Original: In this setting, rich representations of visual inputs and language are learned to automatically generate captions.
Revised: In this setting, rich representations of visual inputs and language are learned to automatically generate captions.

Original: Representations of the latter can be learned in a discriminative monolingual setting, as with ELECTRA (Clark et al., 2020), or in a generative multilingual setting, as with MARGE (Lewis et al., 2020).
Revised: Representations of the latter can be learned in a discriminative monolingual setting, as with ELECTRA (Clark et al., 2020), or in a generative multilingual setting, as with MARGE (Lewis et al., 2020).

Original: However, a discriminative multilingual framework has yet to be explored.
Revised: However, a discriminative multilingual framework has yet to be explored.

Original: Moreover, previous work has focused exclusively on the captioning of medical images (Hasan et al., 2018; Zeng et al., 2020).
Revised: Moreover, previous work has focused exclusively on the captioning of medical images (Hasan et al., 2018; Zeng et al., 2020).

Original: It has not explored the captioning of cardiac time-series signals.
Revised: It has not explored the captioning of cardiac time-series signals.

Original: In this paper, we address the outlined question by designing a neural, multilingual, cardiac signal captioning system while exploiting a large-scale electrocardiogram (ECG) database.
Revised: In this paper, we address the outlined question by designing a neural, multilingual, cardiac signal captioning system while exploiting a large-scale electrocardiogram (ECG) database.

Original: Our contributions are the following: (1) We propose a discriminative multilingual representation learning method, replaced token language prediction (RTLP), in which we randomly replace tokens with those from other languages, and task a network with predicting the language of all tokens.
Revised: Our contributions are the following: (1) We propose a discriminative multilingual representation learning method, replaced token language prediction (RTLP), in which we randomly replace tokens with those from other languages, and task a network with predicting the language of all tokens.

Original: (2) We show that
Revised: (2) We show that

Original: RTLP allows networks to generate reports, in multiple languages, that are more clinically accurate than those generated by masked language modelling (MLM).
Revised: RTLP allows networks to generate reports, in multiple languages, that are as clinically accurate as those generated by masked language modelling (MLM).

Original: To the best of our knowledge, we are the ﬁrst to propose a multilingual cardiac signal captioning system.
Revised: To the best of our knowledge, we are the ﬁrst to propose a multilingual cardiac signal captioning system.

Original: (3) We conduct extensive studies to show that, with RTLP, multilingual learning outperforms and generates more diverse reports  than its monolingual counterpart, a phenomenon we refer to as the blessing of multilinguality .
Revised: (3) We conduct extensive studies to show that generated reports exhibit higher quality and clinical utility when RTLP is ﬁne-tuned in a multilingual setting than in a monolingual setting, a phenomenon we refer to as the blessing of multilinguality .

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Language representation learning Representation learning is integral to modern natural language processing.
Revised: Language representation learning Representation learning is integral to modern natural language processing.

Original: Generative tasks such as MLM have proven effective (Devlin et al., 2019; Liu et al., 2019b), even in the multilingual setting (Conneau & Lample, 2019; Conneau et al., 2020b; Liu et al., 2020).
Revised: Generative tasks such as MLM show promise (Devlin et al., 2019; Liu et al., 2019b), even in the multilingual setting (Conneau & Lample, 2019; Conneau et al., 2020b; Liu et al., 2020).

Original: MARGE (Lewis et al., 2020) retrieves documents, in potentially different languages, and attempts to reconstruct a target document.
Revised: MARGE (Lewis et al., 2020) retrieves documents, in potentially different languages, and attempts to reconstruct a target document.

Original: Others propose to jointly learn textual and visual representations (Sun et al., 2019;
Revised: Others propose to jointly learn textual and visual representations (Sun et al., 2019;

Original: Lu et al., 2019; Zhang et al., 2020b).
Revised: Lu et al., 2019; Zhang et al., 2020b).

Original: Similar to our work is ELECTRA (Clark et al., 2020), where tokens are replaced with those from a generative model, and a network is tasked with discriminating between the original and replaced tokens.
Revised: Similar to our work is ELECTRA (Clark et al., 2020), where tokens are replaced with those from a generative model, and a network is tasked with discriminating between the original and replaced tokens.

Original: This approach, however, is computationally expensive as it requires the learning of both an MLM and discriminative network.
Revised: This approach remains computationally expensive where both an MLM and discriminative network are learned.

Original: Moreover, it does not include the optimization of a multilingual objective.
Revised: It also does not optimize a multilingual objective.

Original: To the best of our knowledge, we are the ﬁrst to propose a discriminative multilingual representation learning method in the context of cardiac signals.
Revised: To the best of our knowledge, we are the ﬁrst to propose a discriminative multilingual representation learning method in the context of cardiac signals.

Original: Multilingual representation learning Pre-training and ﬁne-tuning networks on multiple languages has conferred beneﬁts to NLP tasks (Conneau et al., 2020b; Pratap et al., 2020; Conneau et al., 2020a; Artetxe et al., 2020).
Revised: Multilingual representation learning Pre-training and ﬁne-tuning networks on multiple languages confers beneﬁts to NLP tasks (Conneau et al., 2020b; Pratap et al., 2020; Conneau et al., 2020a; Artetxe et al., 2020).

Original: For example, Conneau et al.
Revised: For example, Conneau et al.

Original: (2020b) and Artetxe et al.
Revised: (2020b) and Artetxe et al.

Original: (2020) show that multilingual pre-training is more advantageous than its monolingual counterpart in solving downstream NLP tasks.
Revised: (2020) show that multilingual pre-training outperforms its monolingual counterpart in solving downstream NLP tasks.

Original: These ﬁndings have been partially driven by monolingual datasets that are machine-translated to multiple languages,such as  XNLI (Conneau et al., 2018).
Revised: These ﬁndings have been partially driven by monolingual datasets that are machine-translated to multiple languages, as with XNLI (Conneau et al., 2018).

Original: We similarly translate ground-truth ECG reports into multiple languages in order to guide the generation of multilingual reports.
Revised: We similarly translate ground-truth ECG reports into multiple languages in order to guide the generation of multilingual reports.

Original: Although Huang et al.
Revised: Although Huang et al.

Original: (2019a) propose a pre-training setup similar to ours, we explore more languages and deﬁne a different pre-training task (RTLP) that is exploited for cardiac signal captioning.
Revised: (2019) propose a pre-training setup similar to ours, we explore more languages and deﬁne a different pre-training task (RTLP) which we exploit for cardiac signal captioning.

Original: Captioning in healthcare Within biomedical image captioning, most research has  focused on chest X-rays (Hasan et al., 2018; Kisilev et al., 2016; Zeng et al., 2020).
Revised: Captioning in healthcare Biomedical image captioning has traditionally focused on chest X-rays (Kisilev et al., 2016; Jing et al., 2017; Hasan et al., 2018; Jing et al., 2020; Zeng et al., 2020; Liu et al., 2021).

Original: Liu et al.
Revised: Liu et al.

Original: (2019a) condition their captioning system on the medical topic to be discussed and Wang et al.
Revised: (2019a) condition their captioning system on the medical topic to be discussed and Wang et al.

Original: (2018) propose a multi-level attention model that attends to both the image and the text.
Revised: (2018) propose a multi-level attention model that attends to both the image and the text.

Original: More recently, methods such as ClinicalBert (Huang et al., 2019b), BioBert (Lee et al., 2020), and BioELMo (Jin et al., 2019) were shown to beneﬁt biomedical applications (Yoon et al., 2019).

Original: Previous work which captioned electroencephalogram (EEG) signals (Biswal et al., 2019; 2020) did not explore a multilingual representation learning method and does not extend to the generation of reports in multiple languages.
Revised: Previous work which captioned electroencephalogram (EEG) signals (Biswal et al., 2019; 2020) did not explore a multilingual representation learning method and does not extend to the generation of reports in multiple languages.

Original: To the best of our knowledge, we are the ﬁrst to propose the multilingual captioning of cardiac signals.
Revised: To the best of our knowledge, we are the ﬁrst to propose the multilingual captioning of cardiac signals.

Original:  B ACKGROUND 3. C ARDIAC S IGNAL C APTIONING
Revised:  B ACKGROUND 3. C ARDIAC S IGNAL C APTIONING

Original: We begin by assuming access to a dataset, D = { x i , cap i } Ni =1 comprising N cardiac signals, x ∈ R D , and their associated captions, cap = { w s } Ss =1 , which consist of S words, w s .
Revised: We begin by assuming access to a dataset, D = { x i , cap i } Ni =1 comprising N cardiac signals, x ∈ R D , and their associated captions, cap = { w s } Ss =1 , which consist of S words, w s .

Original: The goal of cardiac signal captioning is to generate a caption (report) that reliably summarizes the physiological state of a patient as manifested in a cardiac signal.
Revised: The goal of cardiac signal captioning is to generate a caption (report) that reliably summarizes the physiological state of a patient as manifested in a cardiac signal.

Original: To extract features from the signal, an encoder, f θ : x ∈ R D → { v t ∈ R M } Tt =1 , parameterized by θ , maps a D -dimensional instance, x , to a set of
Revised: To extract features from the signal, an encoder, f θ : x ∈ R D → { v t ∈ R M } Tt =1 , parameterized by θ , maps a D -dimensional instance, x , to a set of

Original: T representations, { v t ∈ R M } Tt =1 , each of which is M -dimensional
Revised: T representations, { v t ∈ R M } Tt =1 , each of which is M -dimensional

Original: (see Fig.
Revised: (see Fig.

Original: 1 left).
Revised: 1 left).

Original: To convert the captions into a format ingestible by a network, we ﬁrst convert each word, w s , in a caption, cap = { w s } Ss =1 , to a token, u s , to form a sequence of tokens { u s } Ss =1 .
Revised: To convert the captions into a format ingestible by a network, we ﬁrst convert each word, w s , in a caption, cap = { w s } Ss =1 , to a token, u s , to form a sequence of tokens { u s } Ss =1 .

Original: Such tokenization can involve lower-casing and stemming words, in addition to removing punctuation.
Revised: Such tokenization can involve lower-casing and stemming words, in addition to removing punctuation.

Original: After deriving tokens from all captions in a training set, we form a ﬁxed vocabulary, V = { u i } Ci =1 , of the C unique tokens where | V | =
Revised: After deriving tokens from all captions in a training set, we form a ﬁxed vocabulary, V = { u i } Ci =1 , of the C unique tokens where | V | =

Original: C .
Revised: C .

Original: We then deﬁne an embedding matrix (lookup table),
Revised: We then deﬁne an embedding matrix (lookup table),

Original: E ∈ R C × M : u → e ∈ R M , which maps each token, u , to an M -dimensional token embedding, e .
Revised: E ∈ R C × M : u → e ∈ R M , which maps each token, u , to an M -dimensional token embedding, e .

Original: These embeddings are typically randomly-initialized and learned in an end-to-end manner via gradient descent.
Revised: These embeddings are typically randomly-initialized and learned in an end-to-end manner via gradient descent.

Original: As such, every caption can now be represented as a sequence of token embeddings, { e s } Ss =1 .
Revised: As such, every caption can now be represented as a sequence of token embeddings, { e s } Ss =1 .

Original: To extract features from language, a decoder, g φ : { v t } Tt =1 , { e s } Ss =1 → { h s ∈ R M } Ss =1 , parameterized by φ , attends to all encoder representations, { v t } T t =1 , while mapping a token em-
Revised: To extract features from language, a decoder, g φ : { v t } Tt =1 , { e s } Ss =1 → { h s ∈ R M } Ss =1 , parameterized by φ , attends to all encoder representations, { v t } Tt =1 , while mapping a token embedding, e s , at step, s , to an M -dimensional token representation, h s . Each token representation, h s , in the sequence of representations, { h s } Ss =1 , is then fed into a linear classiﬁcation head, p ω : h s ∈ R M → y s ∈ R C , to output a probability distribution, y s , over the C tokens in the vocabulary. This forms a sequence of outputs, { y s } Ss =1 . At each step, s , in the sequence, the goal is to maximize the likelihood of generating the token of the next step, u s +1 .

Revised: Therefore, by identifying the most probable output token at each step, we can form a sentence of words (i.e., a caption).

Original:  M ETHODS 4. M ULTILINGUAL C APTIONING OF C ARDIAC S IGNALS
Revised:  M ETHODS 4. M ULTILINGUAL C APTIONING OF C ARDIAC S IGNALS

Original: At the surface, a monolingual captioning framework would appear to be sufﬁcient for physicians communicating in a single language.
Revised: At the surface, a monolingual captioning framework would appear to be sufﬁcient for physicians communicating in a single language.

Original: However, the motivation for a multilingual captioning framework, in which reports are simultaneously generated in multiple languages, is threefold.
Revised: However, the motivation for a multilingual captioning framework, in which reports are simultaneously generated in multiple languages, is threefold.

Original: First, a multilingual framework would obviate the cumbersome process of having to train a distinct model for different languages (Conneau et al., 2020a).
Revised: First, a multilingual framework would obviate the cumbersome process of having to train a distinct model for different languages (Conneau et al., 2020a).

Original: Second, recent work has demonstrated the beneﬁts of incorporating additional languages into the learning process (Artetxe et al., 2020).
Revised: Second, recent work has demonstrated the beneﬁts of incorporating additional languages into the learning process (Artetxe et al., 2020).

Original: Lastly, a multilingual framework exhibits greater ﬂexibility than its monolingual counterpart, as it can always be collapsed, during inference, to generate captions from a single language.
Revised: Lastly, a multilingual framework exhibits greater ﬂexibility than its monolingual counterpart, as it can always be collapsed, during inference, to generate captions from a single language.

Original: To enable multilingual captioning, we ﬁrst assume access to L language-speciﬁc datasets, {D l } Ll where D l = { x i , cap li } Ni =1 comprises N cardiac signals, x , and captions in a speciﬁc language, l ∈ L
Revised: To enable multilingual captioning, we ﬁrst assume access to L language-speciﬁc datasets, {D l } Ll where D l = { x i , cap l i } N i =1 comprises N cardiac signals, x , and captions in a speciﬁc language, l ∈ L

Original: = { en , es , · · · } , where en and es represent English and Spanish, respectively.
Revised: = { en , es , · · · } , where en and es represent English and Spanish, respectively.

Original: Note that the cardiac signals are shared across the datasets.
Revised: Note that the cardiac signals are shared across the datasets.

Original: To generate captions in multiple languages, we follow the same encoder-decoder approach mentioned in the previous section with one exception.
Revised: We follow the same encoder-decoder approach mentioned in the previous section with one exception.

Original: Wenow replace the single classiﬁcation head with L language-speciﬁc heads to account for the distinct vocabularies of the L languages.
Revised: We replace the single classiﬁcation head with L language-speciﬁc heads to account for the distinct vocabularies of the L languages.

Original: In doing so, we exploit recent observations that demonstrated the utility of having a network with parameters that are both language-speciﬁc and shared across languages (Zhang et al., 2020a).
Revised: In doing so, we exploit recent observations that demonstrated the utility of having a network with parameters that are both language-speciﬁc and shared across languages (Zhang et al., 2020a).

Original: Formally, for each language, l , we have a linear classiﬁcation head, p ω l : h ls ∈ R M → y ls ∈ R C , with ω l ∈ { ω en , ω es , · · · } reﬂecting language-speciﬁc parameters.
Revised: Formally, for each language, l , we have a linear classiﬁcation head, p ω l : h ls ∈ R M → y ls ∈ R C , with ω l ∈ { ω en , ω es , · · · } reﬂecting language-speciﬁc parameters.

Original: Each head maps the token representation, h ls , at each step, s , in the sequence to a probability distribution, y ls , over C tokens where C ∈ {| V en | , | V es | , · · · } reﬂects the size of a language-speciﬁc token vocabulary.
Revised: Each head maps the token representation, h ls , at each step, s , in the sequence to a probability distribution, y ls , over C tokens where C ∈ {| V en | , | V es | , · · · } reﬂects the size of a language-speciﬁc token vocabulary.

Original: When doing this for each step in the sequence, we arrive at a set of probability distributions, { y ls } Ss =1 .
Revised: When doing this for each step in the sequence, we arrive at a set of probability distributions, { y ls } Ss =1 .

Original: As with traditional language models, at each step in the sequence, s ∈ [1 , S ] , we maximize the likelihood of observing the next token in the sequence, u ls +1 , from a particular language, l ∈ L .
Revised: As with traditional language models, at each step in the sequence, s ∈ [1 , S ] , we maximize the likelihood of observing the next token in the sequence, u ls +1 , from a particular language, l ∈ L .

Original: Therefore, for a mini-batch of B captions in a single language, we would optimize the categorical cross-entropy loss at each step, s .
Revised: Therefore, for a mini-batch of B captions in a single language, we would optimize the categorical cross-entropy loss at each step, s .

Original: To extend this to L languages, we load L mini-batches and optimize the following multi-task categorical cross-entropy loss.
Revised: To extend this to L languages, we load L mini-batches and optimize the following multi-task categorical cross-entropy loss.

Original: [Equation]
Revised: [Equation]

Original: 4. R EPLACED T OKEN L ANGUAGE P REDICTION
Revised: 4. R EPLACED T OKEN L ANGUAGE P REDICTION

Original: To facilitate achieving the downstream task of multilingual cardiac signal captioning, we design a discriminative multilingual pre-training task that learns the decoder parameters, φ , and the token embeddings, e .
Revised: To facilitate achieving the downstream task of multilingual cardiac signal captioning, we design a discriminative multilingual pre-training task that learns the decoder parameters, φ , and the token embeddings, e .

Original: At a high-level, this task involves randomly selecting tokens in a sequence, replacing them with semantically-similar tokens from a different language, and tasking a network with classifying the language of all tokens (see Fig.
Revised: At a high-level, this task involves randomly selecting tokens in a sequence, replacing them with semantically-similar tokens from a different language, and tasking a network with classifying the language of all tokens (see Fig.

Original: 1 right).
Revised: 1 right).

Original: The intuition behind our framework is that a network exposed to semantically-similar tokens from distinct languages which share the same context (neighbouring tokens) can learn that such tokens are indeed similar to one another.
Revised: The intuition behind our framework is that a network exposed to semantically-similar tokens from distinct languages which share the same context (neighbouring tokens) can learn that such tokens are indeed similar to one another.

Original: We hypothesize that encouraging this behaviour can lead to the learning of token representations that are beneﬁcial for the downstream task of multilingual captioning since these tokens (from different languages) are likely to arise in the same generated report, albeit in a different language.
Revised: We hypothesize that encouraging this behaviour can lead to the learning of token representations that are beneﬁcial for the downstream task of multilingual captioning since these tokens (from different languages) are likely to arise in the same generated report, albeit in a different language.

Original: As such, the generated multilingual reports might be more likely to contain plausible tokens, and thus be clinically accurate.
Revised: As such, the generated multilingual reports might be more likely to contain plausible tokens, and thus be clinically accurate.

Original: From hereon forward, we refer to this method as replaced token language prediction (RTLP) and describe its mechanics next.
Revised: From hereon forward, we refer to this method as replaced token language prediction (RTLP) and describe its mechanics next.

Original: Source token selection Given tokens, { u l src s } Ss =1 , in a sequence of length, S , from a source language, l src ∈ L , we ﬁrst sample K distinct steps, { s k } Kk =1 where s k ∈ [1 , S ] from a uniform distribution, U .
Revised: Source token selection Given tokens, { u l src s } Ss =1 , in a sequence of length, S , from a source language, l src ∈ L , we ﬁrst sample K distinct steps, { s k } Kk =1 where s k ∈ [1 , S ] from a uniform distribution, U .

Original: We then replace the corresponding source tokens, { u l src s k } Kk =1 , with those from a target language, { u l tgt s k } Kk =1 .
Revised: We then replace the corresponding source tokens, { u l src s k } Kk =1 , with those from a target language, { u l tgt s k } Kk =1 .

Original: We outline how to select the target language and token next.
Revised: We outline how to select the target language and token next.

Original: Target language and token selection For each source token, u l src s k , we sample a target language, l tgt ∼ U
Revised: Target language and token selection For each source token, u l src s k , we sample a target language, l tgt ∼ U

Original: ( L (cid:48) ) , uniformly at random from the set of remaining languages L (cid:48) =
Revised: ( L (cid:48) ) , uniformly at random from the set of remaining languages L (cid:48) =

Original: L \ l src where | L (cid:48) | =
Revised: L \ l src where | L (cid:48) | =

Original: L − 1 .
Revised: L − 1 .

Original: Given l tgt , we now sample a target token, u l tgt s k ∼ U
Revised: Given l tgt , we now sample a target token, u l tgt s k ∼ U

Original: ( V l tgt ) , uniformly at random from the language-speciﬁc vocabulary of tokens, V l tgt .
Revised: ( V l tgt ) , uniformly at random from the language-speciﬁc vocabulary of tokens, V l tgt .

Original: Such random sampling, however, can lead to the selection of a target token that is semantically different from the source token.
Revised: Such random sampling, however, can lead to the selection of a target token that is semantically different from the source token.

Original: As such, the network may discriminate between source and target tokens by using a detrimental shortcut that is based on semantics instead of language.
Revised: As such, the network may discriminate between source and target tokens by using a detrimental shortcut that is based on semantics instead of language.

Original: To avoid this behaviour, we instead adopt a strategy where the target token is likely to be semantically similar to (e.g., a noisy translation of) the source token.
Revised: To avoid this behaviour, we instead adopt a strategy where the target token is likely to be semantically similar to (e.g., a noisy translation of) the source token.

Original: Formally, we quantify the cosine similarity, sim j , between the source token embedding, e l src s k ∈ R M , and the embedding, e l tgt j ∈ R M of each token, u l tgt j ∈ V l tgt in the vocabulary of the target language.
Revised: Formally, we quantify the cosine similarity, sim j , between the source token embedding, e l src s k ∈ R M , and the embedding, e l tgt j ∈ R M of each token, u l tgt j ∈ V l tgt in the vocabulary of the target language.

Original: By taking the softmax of these similarities, we form a categorical distribution, q , with elements, q j , from which we sample u l tgt s k .
Revised: By taking the softmax of these similarities, we form a categorical distribution, q , with elements, q j , from which we sample u l tgt s k .

Original: As the token embeddings become more meaningful during training, the sampled target token is more likely to be semantically similar to the source token. 
Revised: As the token embeddings become more meaningful during training, the sampled target token is more likely to be semantically similar to the source token. [Equation] where

Original: [Equation]
Revised: [Equation]

Original: Objective function Equipped with a sequence of tokens in source and target languages, we predict the language of such tokens.
Revised: Objective function Equipped with a sequence of tokens in source and target languages, we predict the language of such tokens.

Original: To do so, we deﬁne a classiﬁcation head, p ψ : h → y ∈ R L , parameterized by ψ , that maps each token representation, h , to a probability distribution, y , over L languages.
Revised: To do so, we deﬁne a classiﬁcation head, p ψ : h → y ∈ R L , parameterized by ψ , that maps each token representation, h , to a probability distribution, y , over L languages.

Original: Formally, given a mini-batch of B captions of length, S , in the source language, l , we optimize the categorical cross-entropy loss.
Revised: Formally, given a mini-batch of B captions of length, S , in the source language, l , we optimize the categorical cross-entropy loss.

Original: This is achieved for the tokens in the source ( r = 0 ) and target ( r = 1 ) languages.
Revised: This is achieved for the tokens in the source ( r = 0 ) and target ( r = 1 ) languages.

Original: To extend this to L source languages, we load L mini-batches and optimize the following multi-task categorical cross-entropy loss where 1 is the indicator function.
Revised: To extend this to L source languages, we load L mini-batches and optimize the following multi-task categorical cross-entropy loss where 1 is the indicator function.

Original: [Equation]
Revised: [Equation]

Original:  E XPERIMENTAL D ESIGN
Revised:  E XPERIMENTAL D ESIGN

Original: Datasets We evaluate our framework on the PTB-XL dataset (Wagner et al., 2020), the only publicly-available dataset which comprises ECG signals with a corresponding clinical report.
Revised: Datasets We evaluate our framework on the PTB-XL dataset (Wagner et al., 2020), the only publicly-available dataset which comprises ECG signals with a corresponding clinical report.

Original: Each
Revised: Each

Original: ECG signal (from a total of 18,885 patients) is also associated with cardiac arrhythmia labels which we group into 5 classes (Strodthoff et al., 2020).
Revised: ECG signal (from a total of 18,885 patients) is also associated with cardiac abnormality labels which we group into 5 classes (Strodthoff et al., 2020).

Original: We provide further details in Appendix A.1.
Revised: We provide further details in Appendix A.1.

Original: Representation learning of cardiac signals Supervised pre-training continues to be an effective way to learn rich, generalizable representations.
Revised: Representation learning of cardiac signals Supervised pre-training remains an effective way to learn rich, generalizable representations.

Original: To that end, we pre-train the encoder , f θ , to map 12lead ECG signalsto abnormalities in the functioning of the heart (cardiac arrhythmias).
Revised: As such, we pre-train the encoder , f θ , to map 12-lead ECG signals in the training set to cardiac abnormalities.

Original: This choice is motivated by our observation that ECG reports are likely to reﬂect the pathology of a cardiac arrhythmia.
Revised: We decide on this task because ECG reports are likely to reﬂect such abnormalities.

Original: Therefore, learning representations of cardiac signals that can discriminate between cardiac arrhythmias may be beneﬁcial for the downstream task of cardiac signal captioning.
Revised: Therefore, grounding the captioning process in representations of cardiac signals that can discriminate between cardiac abnormalities can be beneﬁcial.

Original: Representation learning of clinical reports To implement RTLP, we exploit L language-speciﬁc tokenizers from SpaCy to tokenize the ECG reports.
Revised: Representation learning of clinical reports To tokenize the ECG reports, we exploit L languagespeciﬁc tokenizers from SpaCy (see Appendix C.5 for details).

Original: For simplicity, we lower-case the text and remove any punctuation.
Revised: For simplicity, we lower-case the text and remove any punctuation.

Original: By keeping track of unique tokens, we form L distinct vocabularies.
Revised: By keeping track of unique tokens, we form L distinct vocabularies.

Original: Each vocabulary also includes language-speciﬁc tokens to indicate the start and end of the report, and the [PAD] and [OOV] tokens to refer to padded entries and tokens observed during inference that are not seen during training, respectively.
Revised: Each vocabulary also includes language-speciﬁc tokens to indicate the start and end of the report, and the [PAD] and [OOV] tokens to refer to padded entries and tokens observed during inference that are not seen during training, respectively.

Original: We also introduce the [MASK] token where appropriate.
Revised: We also introduce the [MASK] token where appropriate.

Original:  Multilingual cardiac signal captioning After pre-training the encoder, f θ , and the decoder, g φ , independently of one another, we exploit the learned parameters, { θ, φ } , and token embeddings to solve the task of cardiac signal captioning (see Fig.
Revised: The details of network architectures can be found in Appendix C.1. Multilingual cardiac signal captioning After pre-training the encoder, f θ , and the decoder, g φ , independently of one another, we exploit the learned parameters, { θ, φ } , and token embeddings to solve the task of cardiac signal captioning (see Fig.

Original: 1 left).
Revised: 1 left).

Original: After experimenting with several variants of our framework, we chose to freeze the encoder parameters and extract multiple representations per cardiac signal.
Revised: After experimenting with several variants of our framework, we chose to freeze the encoder parameters and extract multiple representations per cardiac signal.

Original: These encoder representations are used in the cross-attention mechanism of the Transformer decoder.

Original: Multilingual captioning also requires a ground-truth ECG report to be available in multiple languages.
Revised: Multilingual captioning also requires multilingual ground-truth ECG reports.

Original: Since such paired reports do not exist, we translate the original set of reports in English (en) to six languages { German (de) , Greek (el) , Spanish (es) , French (fr) , Italian (it) , Portuguese (pt) } using the Google Translate API, a strategy similar to that adopted by Conneau et al.
Revised: Since such paired reports do not exist, we translate the original set of reports in English (en) to six languages { German (de) , Greek (el) , Spanish (es) , French (fr) , Italian (it) , Portuguese (pt) } using the Google Translate API, a strategy similar to that adopted by Conneau et al.

Original: We open-source these reports 1 and provide further details in Appendix B.
Revised: We opensource these reports 1 and provide further details in Appendix B.

Original: Although translated reports can be imperfect ground-truth reports, we hypothesize (and indeed show) that the net effect of multilinguality on downstream performance is advantageous.
Revised: Although translated reports can be imperfect ground-truth reports, we hypothesize (and indeed show) that the net effect of multilinguality on the quality and clinical utility of generated reports is advantageous.

Original: Baseline Methods Aswe are the ﬁrst to proposethe task of multilingual captioning of cardiac signals,  previous methods do not exist for comparison.
Revised: Baseline Methods As the ﬁrst to propose multilingual captioning of cardiac signals, we cannot trivially compare to previous methods.

Original: However, we compare our pre-training method, RTLP, to the following: 1) MLM (Devlin et al., 2019),a masked language modellingpre-training objective where the decoder is tasked with identifying masked tokens, 2) ELECTRA (Clark et al., 2020),a pre-training objective where the decoder is tasked with identifying whether tokens have been replaced with those from an MLM model, and 3) MARGE (Lewis et al., 2020), a multilingual generative language representation learning approach where source documents in various languages are retrieved to generate a similaryet distinct target document (see Appendix C. formore details on how we adapted these methods).
Revised: However, we compare our pre-training method, RTLP, to the following state-of-the-art pre-training methods: 1) MLM (Devlin et al., 2019), masked language modelling where the decoder is tasked with identifying masked tokens, 2) ELECTRA (Clark et al., 2020), where the decoder is tasked with identifying whether tokens have been replaced with those from an MLM model, and 3) MARGE (Lewis et al., 2020), a multilingual generative language representation learning approach where source documents in various languages are retrieved to generate a similar target document (see Appendix C.3 for details on how we adapted these methods).

Original: Evaluation Metrics We quantify the quality of the generated reports by comparing the degree of overlap of tokens in such reports to those in the ground-truth report.
Revised: Evaluation Metrics We quantify the quality of the generated reports by comparing the degree of overlap of tokens in such reports to those in the ground-truth report.

Original: Speciﬁcally, we use the
Revised: Speciﬁcally, we use the

Original: BLEU − 1 (Papineni et al., 2002), METEOR (Banerjee & Lavie, 2005), and ROUGE − L (Lin, 2004) scores (see Appendix C.4 for more details).
Revised: BLEU − 1 (Papineni et al., 2002), METEOR (Banerjee & Lavie, 2005), and ROUGE − L (Lin, 2004) scores (see Appendix C.4 for more details).

Original: Hyperparameters We conduct our experiments using PyTorch (Paszke et al., 2019).
Revised: Hyperparameters We conduct our experiments using PyTorch (Paszke et al., 2019).

Original: We pre-train the encoder and decoder with a patience value of 10 and 25 epochs, respectively, on the validation loss.
Revised: We pre-train the encoder and decoder with a patience value of 10 and 25 epochs, respectively, on the validation loss.

Original: When transferring the parameters to the task of cardiac signal captioning, we use those associated with the lowest validation loss.
Revised: When transferring the parameters to the task of cardiac signal captioning, we use those associated with the lowest validation loss.

Original: When ﬁne-tuning, we checkpoint the parameters associated with the highest validation BLEU score.
Revised: When ﬁne-tuning, we checkpoint the parameters associated with the highest validation BLEU score.

Original: We implement a greedy decoding mechanism by taking the argmax of the probability distribution over tokens at each step.
Revised: We implement a greedy decoding mechanism by taking the argmax of the probability distribution over tokens at each step.

Original: We leave for future work other decoding mechanisms such as beam search (Graves, 2012).
Revised: We leave for future work other decoding mechanisms such as beam search (Graves, 2012).

Original:  E XPERIMENTAL R ESULTS 6. Q UANTITATIVE E VALUATION OF G ENERATED R EPORTS
Revised:  E XPERIMENTAL R ESULTS 6. Q UANTITATIVE E VALUATION OF G ENERATED R EPORTS

Original: We begin by quantitatively evaluating the ability of the pre-training methods to generate high quality multilingual reports.
Revised: We begin by quantitatively evaluating the ability of the pre-training methods to generate high quality multilingual reports.

Original: In Table 1, we present the BLEU − 1 , METEOR, and ROUGE − L scores of reports generated in seven different languages.
Revised: In Table 1, we present the BLEU − 1 , METEOR, and ROUGE − L scores of reports generated in seven different languages.

Original: We ﬁnd that RTLP performs on par with state-of-the-art generative pre-training methods.
Revised: We ﬁnd that RTLP performs on par with state-of-the-art generative pre-training methods.

Original: For example, on average, RTLP achieves BLEU − 1 = 28 .
Revised: For example, on average, RTLP achieves BLEU − 1 = 28 .

Original: 5 whereas MLM and MARGE achieve
Revised: 5 whereas MLM and MARGE achieve

Original: BLEU − 1 = 29 . 4 and 28 . 9 , respectively.
Revised: BLEU − 1 = 29 . 4 and 28 . 9 , respectively.

Original: This ﬁnding holds across languages and evaluation metrics.
Revised: This ﬁnding holds across languages and evaluation metrics.

Original: Furthermore, we ﬁnd that RTLP outperforms the state-of-the-art discriminative pre-training method, ELECTRA.
Revised: Furthermore, we ﬁnd that RTLP outperforms the state-of-the-art discriminative pre-training method, ELECTRA.

Original: For example, on average, RTLP and ELECTRA achieve ROUGE − L = 33 .and 0 .
Revised: For example, on average, RTLP and ELECTRA achieve ROUGE − L = 33 .and 0 .

Original: 5 , respectively.
Revised: 5 , respectively.

Original: We note that we were unable to achieve satisfactory performance with ELECTRA despite extensive experimentation (please see GitHub for reproducibility).
Revised: We note that we were unable to achieve satisfactory performance with ELECTRA despite extensive experimentation (please see code for reproducibility).

Original: One hypothesis for this stems from the difﬁculty of optimizing its objective function which comprises a generative and discriminative term.
Revised: One hypothesis for this stems from the difﬁculty of optimizing its objective function which comprises a generative and discriminative term.

Original: We also ﬁnd that, regardless of the pre-training method implemented, performance varies signiﬁcantly across languages.
Revised: We also ﬁnd that, regardless of the pre-training method implemented, performance varies signiﬁcantly across languages.

Original: For example, RTLP achieves BLEU − 1 = 19 .
Revised: For example, RTLP achieves BLEU − 1 = 19 .

Original: 8 and 33 . 5 on Greek (el) and English (en) reports, respectively.
Revised: 8 and 33 . 5 on Greek (el) and English (en) reports, respectively.

Original: We hypothesize that this is due to a high level of dissimilarity between the Greek vocabulary and that of the remaining languages.
Revised: We hypothesize that this is due to a high level of dissimilarity between the Greek vocabulary and that of the remaining languages.

Original: This, in turn, reduces the amount of knowledge transferred from the other languages to the Greek language.
Revised: This, in turn, reduces the amount of knowledge transferred from the other languages to the Greek language.

Original: Overall, our ﬁndings suggest that RTLP can be an effective discriminative multilingual pre-training method.
Revised: Overall, our ﬁndings suggest that RTLP can be an effective discriminative multilingual pre-training method.

Original: Qualitative evidence to support this claim is provided in the next section.
Revised: Qualitative evidence to support this claim is provided in the next section.

Original: [Table]
Revised: [Table]

Original: 6. Q UALITATIVE E VALUATION OF G ENERATED R EPORTS
Revised: 6. Q UALITATIVE E VALUATION OF G ENERATED R EPORTS

Original: We also manually inspect the multilingual clinical reports generated by RTLP.
Revised: We also manually inspect the multilingual clinical reports generated by RTLP.

Original: Ideally, such reports should accurately reﬂect clinical information.
Revised: Ideally, such reports should accurately reﬂect clinical information.

Original: In Fig
Revised: In Fig

Original: [Table]
Revised: [Table]

Original: we present a 12-lead ECG segment alongside the multilingual ground-truth reports and those generated by RTLP and MLM
Revised: we present a 12-lead ECG segment alongside the multilingual ground-truth reports and those generated by RTLP and MLM

Original: [Table]
Revised: [Table]

Original: There are three main takeaways from Fig 2.
Revised: There are three main takeaways from Fig 2.

Original: First, RTLP allows networks to generate reports that accurately capture the aberrant morphology (shape) of the ECG signal.
Revised: First, RTLP allows networks to generate reports that accurately capture the aberrant morphology (shape) of the ECG signal.

Original: For example, in Spanish (es), the ground-truth and RTLP-generated reports both explicitly mention the cardiac abnormality “bloqueo de rama izquierda” (left bundle branch block).
Revised: For example, in Spanish (es), the ground-truth and RTLP-generated reports both explicitly mention the cardiac abnormality “bloqueo de rama izquierda” (left bundle branch block).

Original: Second, we ﬁnd that RTLP-generated reports manage to capture critical aspects of the ECG signal that their MLM counterparts struggle with.
Revised: Second, we ﬁnd that RTLP-generated reports manage to capture critical aspects of the ECG signal that their MLM counterparts struggle with.

Original: For example, in English (en), both the ground-truth and RTLP-generated reports both explicitly mention “left hypertrophy possible” , whereas this phrase is noticeably absent from the MLM-generated report.
Revised: For example, in English (en), both the ground-truth and RTLP-generated reports both explicitly mention “left hypertrophy possible” , whereas this phrase is noticeably absent from the MLM-generated report.

Original: Such an absence is problematic as it might result in a physician overlooking this aspect, and thus failing to act accordingly.
Revised: Such an absence is problematic as it might result in a physician overlooking this aspect, and thus failing to act accordingly.

Original: Lastly, we ﬁnd that MLM-generated reports can include additional erroneous phrases which are neither present in the ground-truth report nor in the RTLP-generated report.
Revised: Lastly, we ﬁnd that MLM-generated reports can include additional erroneous phrases which are neither present in the ground-truth report nor in the RTLP-generated report.

Original: For example, in Portuguese (pt), the MLM-generated report mentions “atrial ﬂutter” , which is noticeably (and correctly) absent from the remaining reports.
Revised: For example, in Portuguese (pt), the MLM-generated report mentions “atrial ﬂutter” , which is noticeably (and correctly) absent from the remaining reports.

Original: Similar erroneous inclusions can also be found in the German (de) and French (fr) reports.
Revised: Similar erroneous inclusions can also be found in the German (de) and French (fr) reports.

Original: This is also problematic since physicians can be misled by such statements, potentially resulting in unnecessary medical treatments.
Revised: This is also problematic since physicians can be misled by such statements, potentially resulting in unnecessary medical treatments.

Original: These ﬁndings indicate that RTLP allows networks to generate reliable and clinically accurate reports.
Revised: These ﬁndings indicate that RTLP allows networks to generate reliable and clinically accurate reports.

Original: 6. Q UANTIFICATION OF THE D IVERSITY OF G ENERATED R EPORTS
Revised: 6. Q UANTIFICATION OF C LINICAL U TILITY OF G ENERATED R EPORTS

Original: So far, we have  shown that RTLP is capable of generating clinically accurate and plausible reports.
Revised: So far, we have qualitatively shown that RTLP is capable of generating clinically accurate and plausible reports.

Original: It could be argued that generated reports should also exhibit a certain level of diversity in text (Li &
Revised: It could be argued, however, that such a qualitative evaluation presents a limited view of the clinical utility of these reports. More broadly, our evaluation setup can be questioned since we are comparing generated reports to potentially noisy multilingual “ground-truth” reports, which were, for the most part, generated via translation.

Original: Jurafsky, 2016). To illustrate why, consider a constrained system thatsimply regurgitates the same caption regardless of the input cardiac signal.
Revised: In this section, we aim to allay this concern and demonstrate the clinical utility of our generated reports in a systematic manner, as explained next. To demonstrate that the translated ground-truth reports exhibit clinically useful information, we examine their ability to identify characteristics (e.g., cardiac abnormalities) of the corresponding cardiac signal.

Original: Such a system would be of minimal clinical value due to its inability to distinguish between the nuances of the cardiac signals.
Revised: The intuition is that a report which is highly predictive of such characteristics, which inform clinical decision-making, is likely to be of clinical utility.

Original: To quantify the diversity of generated reports, we exploit the Self-BLEU metric (Zhu et al., 2018), which measures the BLEU score between all pairs of generated reports. Intuitively, the lower the value of this metric ( ↓ SelfBLEU), the higher the diversity of the  reports ( ↑ diversity).
Revised: For example, this report could be used for decision support. As such, we set out to learn a model that maps translated ground-truth reports to a single cardiac abnormality label (5 classes) associated with the ECG. We represent these reports as a bag-of-words (BoW) on which a Random Forest model is trained (see Appendix D for further details).

Original: In Fig.
Revised: In Fig.

Original: 3, we present the Self-BLEU score for three differentpre-training methods across all languages in the multilingual setting.
Revised: 3, we illustrate the performance of such models when trained and evaluated on mutually-exclusive ground-truth reports in different languages ( Target ).

Original: There are several takeaways from Fig. 3.
Revised: We ﬁnd that groundtruth reports are indeed highly predictive of cardiac abnormalities. This is evident by the strong performance of these models (e.g., AUC > 0 . 90 ). Such a ﬁnding suggests that the translated ground-truth reports are of clinical value. To demonstrate the clinical utility of the generated reports, we follow a procedure similar to the one just described with one exception.

Original: First, the degree of diversity exhibited by  the  generated reports differs across the languages.
Revised: Instead of training a model on ground-truth reports, we do so on held-out reports that are generated by a framework (e.g., RTLP). Importantly, we evaluate all models on the same set of ground-truth reports. The intuition here is that high quality generated reports should allow for the learning of a predictive model that can generalize to the ground-truth reports.

Original: For example, the French (fr) reports, on average, exhibit the lowest Self-BLUE ( ≈ 30 ) and thus highest diversity whereas the Spanish (es) reports exhibit the least diversity (Self-BLEU ≈ 40 ).
Revised: In Fig. 3, we illustrate the performance of such models when trained on MLM- and RTLPgenerated reports. There are two main ﬁndings here. First, we see that the generated reports exhibit lower clinical utility than ground-truth reports.

Original: We hypothesize that this could be due to the distinct vocabulary sets of these respective languages.
Revised: This is evident by the ↓ AUC when comparing the Target setting to the MLM setting. For example, in German (de), AUC ≈ 0 . 92 → 0 . 82 . Second, we show that the generated reports are predictive of cardiac abnormalities. This is evident by ↑ AUC of models trained on such reports.

Original: Second, we ﬁnd that all three methods, MLM, MARGE, and
Revised: For example, in Italian (it), RTLP-generated reports lead to AUC ≈ 0 . 84 . Such a ﬁnding supports the claim that RTLP-generated reports are of clinical value. Models are evaluated on a mutually-exclusive set of ground-truth reports across ﬁve random seeds.

Original: RTLP, generate reports with a similar level ofdiversity. For example, when generating English (en) reports, these methods achieve Self-BLEU ≈ 35 .
Revised: Translated ground-truth reports and those generated, for example, by RTLP are predictive of cardiac abnormalities. Such a ﬁnding demonstrates the clinical utility of reports, and by extension, the system that generates them.

Original: 6. I NVESTIGATION OF THE C URSE OF M ULTILINGUALITY
Revised: 6. I NVESTIGATION OF THE C URSE OF M ULTILINGUALITY

Original: Multilingual neural systems can experience the curse of multilinguality (Conneau et al., 2020b).
Revised: Multilingual neural systems can experience the curse of multilinguality (Conneau et al., 2020b).

Original: Concisely, this attributes the potentially poorer performance of multilingual models relative to their monolingual counterparts to interference between the various languages.
Revised: Concisely, this attributes the potentially poorer performance of multilingual models relative to their monolingual counterparts to interference between the various languages.

Original: Intuitively, tasking a network with generating reports in multiple languages, analogous to multi-task learning (Caruana, 1993), can be too demanding and thus hinder its ability to generate sensible reports.
Revised: Intuitively, tasking a network with generating reports in multiple languages, analogous to multi-task learning (Caruana, 1993), can be too demanding and thus hinder its ability to generate sensible reports.

Original: We explore this curse from the lens of the clinical accuracy and diversity ofthe generated reports.
Revised: We explore this curse through the lens of the quality and clinical utility of generated reports.

Original: To do so, we ﬁrst pre-train our networks, as per usual, and ﬁne-tune them in the monolingual setting.
Revised: To do so, we ﬁrst pre-train our networks, as per usual, and ﬁne-tune them in the monolingual setting.

Original: We then compare the performance of networks and the diversity of the generated reports in the multilingual setting to those in the monolingual setting.
Revised: We then compare the quality ( BLEU − 1 ) and clinical utility ( AUC ) of the generated reports in the multilingual setting to those in the monolingual setting.

Original: In Fig. 4, we present such a comparison, illustrating
Revised: Such a comparison is presented in Fig.

Original: BLEU − 1 (top row) and Self-BLEU (bottom row) of reports generated by MARGE and RTLP.

Original: In Fig.
Revised: In Fig.

Original: 4 (top row), we ﬁnd that MARGE does not experience the curse of multilinguality when evaluated along the dimension of performance.
Revised: 4 (left column), we ﬁnd that MARGE does not experience the curse of multilinguality when evaluated along the dimension of either performance or clinical utility.

Original: This can be seen by the similar performance achieved by MARGE irrespective of whether it is ﬁne-tuned in the monolingual or multilingual setting.
Revised: This can be seen by the similar performance achieved by MARGE irrespective of whether it is ﬁne-tuned in the monolingual or multilingual setting.

Original: For example, when generating Spanish (es) reports, MARGE achieves BLEU − 1 ≈ 32  in both settings.
Revised: For example, when generating Spanish (es) reports, MARGE achieves BLEU − 1 ≈ 32 and AUC ≈ 0 . 80 in both settings.

Original: Such a ﬁnding suggests that incorporating additional languages into the ﬁne-tuning process has little to no effect on the quality of the generated reports.
Revised: Such a ﬁnding suggests that incorporating multiple languages into the ﬁne-tuning process has little to no effect on the quality and clinical utility

Revised: Through the lens of report quality and clinical utility, MARGE does not consistently experience the curse of multilinguality.

Original: In contrast,we ﬁnd that RTLP beneﬁts signiﬁcantly from  the inclusion of multiple languages.
Revised: In contrast, RTLP beneﬁts signiﬁcantly from incorporating multiple languages into the ﬁne-tuning process, a phenomenon we refer to as the blessing of multilinguality . of the generated reports. This is in contrast to ﬁndings with RTLP (Fig. 4 right column), which reﬂect a signiﬁcant beneﬁt from the inclusion of multiple languages.

Original: This can be seen by the higher generalization performance ( ↑ BLEU − 1 ) achieved by RTLP in the multilingual setting than in the monolingual setting.
Revised: This can be seen by the higher quality ( ↑ BLEU − 1 ) and clinical utility ( ↑ AUC ) of reports generated by RTLP in the multilingual setting than in the monolingual setting.

Original: For example, for Spanish (es) reports, the network in both settings achieves BLEU − 1≈and ≈ 18 , respectively.
Revised: For example, for French (fr) reports and in both settings, the network achieves BLEU − 1 ≈ 30 and ≈ 2 , respectively, and AUC ≈ 0 .

Original: This reﬂects almost a two-fold improvement.
Revised: 80 and 0 . 52 , respectively.

Original: We denote this marginal beneﬁt of multilinguality  as the blessing of multilinguality .
Revised: We denote this positive effect of multilinguality on the quality and clinical utility of reports as the blessing of multilinguality .

Original: We hypothesize that such beneﬁts stem from the transfer and sharing of knowledge across languages.
Revised: We hypothesize that such beneﬁts stem from the transfer and sharing of knowledge across languages.

Original: We also note the poorer performance of RTLP in the monolingual setting relative to the multilingual setting.
Revised: We also note the poorer performance of RTLP in the monolingual setting relative to the multilingual setting.

Original: For English (en) reports, RTLP achieves BLEU − 1 ≈ 13 and ≈ 31 , respectively.
Revised: For English (en) reports, RTLP achieves BLEU − 1 ≈ 13 and ≈ 31 , respectively.

Original: We hypothesize that this is due to the relativedegree of importance placed by the model on particular languages during multilingual pretraining with RTLP.
Revised: We hypothesize that this is due to the relative importance placed by the model on particular languages during multilingual RTLP pre-training.

Original: In other words, pre-training may implicitly weight languages differently.
Revised: For example, such pre-training may implicitly weight languages differently.

Original: As such, the learned language-speciﬁc token representations may differ in their expressiveness.
Revised: As such, the learned language-speciﬁc token representations may differ in their expressiveness.

Original: In light of this, networks ﬁne-tuned in the monolingual setting may perform poorly.In Fig.
Revised: In light of this, networks ﬁne-tuned in the monolingual setting may perform poorly.

Original: 4 (bottom row)

Original: , we ﬁnd that, on average, MARGE generates reports with a similar level of diversity irrespective of whether it is ﬁne-tuned in the monolingual or multilingual setting.

Original: For example, although MARGE generates more diverse ( ↓ Self-BLEU) German (de) reports in the latter setting, the opposite holds for French (fr) reports.

Original: In contrast, RTLP consistently generates more diverse reports ( ↓ Self-BLEU) in the multilingual setting than in the monolingual setting.

Original: For example, when generating English (en) reports, RTLP achieves Self-BLEU

Original: [-≈ 
-]
Revised: [-≈ 
-]

Original: and ≈ 52 , respectively.

Original: Such a ﬁnding also contributes to the blessing of multilinguality .

Original:  D ISCUSSION
Revised:  D ISCUSSION

Original: In this paper, we proposed a neural multilingual cardiac signal captioning framework.
Revised: In this paper, we proposed a neural multilingual cardiac signal captioning framework.

Original: Motivated by the empirical beneﬁts of multilingual representation learning, we designed a discriminative multilingual pre-training method, entitled RTLP. This involved randomly replacing tokens in a caption with those from a different language and tasking the network with identifying the language of all tokens.
Revised: In the process, we designed a discriminative multilingual pre-training method, RTLP, which randomly replaced tokens in a caption with those from a different language and tasked the network with identifying the language of all tokens.

Original: When evaluated in the multilingual setting, RTLP performed on par with  state-of-the-art pre-training methods such as MLM and MARGE.
Revised: We showed that RTLP-generated reports exhibit high quality and clinical utility, and are on par with those generated by state-of-the-art pre-training methods such as MLM.

Original: We also showed that RTLP-generated reports correctly capture the pathology (disease) of ECG signals and are more reliable than those generated by MLM. Lastly, we found that with RTLP, multilingual ﬁne-tuning is preferable to its monolingual counterpart, an observation we termed the blessing of multilinguality.
Revised: We also showed that generated reports exhibit higher quality and clinical utility when RTLP is ﬁne-tuned in a multilingual setting than in a monolingual setting, a phenomenon we refer to as the blessing of multilinguality.

Original: Based on these ﬁndings, we recommend exploiting RTLP for multilingual cardiac signal captioning.
Revised: Based on these ﬁndings, we recommend exploiting RTLP for multilingual cardiac signal captioning.

Original: Looking forward, several exciting paths remain.
Revised: Several exciting paths remain.

Original: For example, designing a captioning framework that incorporates multiple modalities (e.g., medical videos, electronic health record data) from different medical domains may mimic a more realistic clinical setting.
Revised: These include the design of a captioning framework that incorporates multiple modalities (e.g., medical videos, electronic health records).

Original: As such, the framework may generate reports of a higher quality  and increase its likelihood of  adoption by medical professionals.
Revised: Doing so may result in higher quality reports and increase the likelihood of system adoption by medical professionals.

Original: Furthermore, generating reports of a higher level of complexity would be a suitable next step to validate the applicability of our framework.
Revised: In order to further validate our framework, we aim to exploit it to generate reports of a higher level of complexity.

Original: A D ATASETS A. D ATA P REPROCESSING
Revised: A D ATASETS A. D ATA P REPROCESSING

Original: The ECG frames consisted of 2500 samples and consecutive frames had no overlap with one another.
Revised: The ECG frames consisted of 2500 samples and consecutive frames had no overlap with one another.

Original: Data splits were always performed at the patient-level.
Revised: Data splits were always performed at the patient-level.

Original: PTB-XL (Wagner et al., 2020).
Revised: PTB-XL (Wagner et al., 2020).

Original: Each ECG recording was originally 10 seconds with a sampling rate of 500Hz.
Revised: Each ECG recording was originally 10 seconds with a sampling rate of 500Hz.

Original: We extract 5-second non-overlapping segments of each recording generating frames of length 2500 samples.
Revised: We extract 5-second non-overlapping segments of each recording generating frames of length 2500 samples.

Original: We follow the diagnostic class labelling setup suggested by Strodthoff et al.
Revised: We follow the diagnostic class labelling setup suggested by Strodthoff et al.

Original: (2020) which resulted in ﬁve classes: Conduction Disturbance (CD), Hypertrophy (HYP), Myocardial Infarction (MI), Normal (NORM), and Ischemic ST-T Changes (STTC).
Revised: (2020) which resulted in ﬁve classes: Conduction Disturbance (CD), Hypertrophy (HYP), Myocardial Infarction (MI), Normal (NORM), and Ischemic ST-T Changes (STTC).

Original: Furthermore, we only consider ECG segments with one label assigned to them.
Revised: Furthermore, we only consider ECG segments with one label assigned to them.

Original: The ECG frames were standardized to follow a standard Gaussian distribution.
Revised: The ECG frames were standardized to follow a standard Gaussian distribution.

Original: A. D ATA S AMPLES
Revised: A. D ATA S AMPLES

Original: In this section, we outline the number of instances used during training.
Revised: In this section, we outline the number of instances used during training.

Original: [Table]
Revised: [Table]

Original: A. V OCABULARY T OKENS
Revised: A. V OCABULARY T OKENS

Original: In this section, we outline the number of language-speciﬁc tokens available in each language’s vocabulary for the two datasets.
Revised: In this section, we outline the number of language-speciﬁc tokens available in each language’s vocabulary for the two datasets.

Original: [Table]
Revised: [Table]

Original: B T RANSLATION D ETAILS
Revised: B T RANSLATION D ETAILS

Original: In this section, we outline the steps taken to translate the ECG reports originally found in the PTBXL dataset.
Revised: In this section, we outline the steps taken to translate the ECG reports originally found in the PTBXL dataset.

Original: We remind readers that although these ECG reports are a mixture of English and German, they are predominantly in the latter.
Revised: We remind readers that although these ECG reports are a mixture of English and German, they are predominantly in the latter.

Original: As a result, we treat German as the source language from which we translate the reports to other languages.
Revised: As a result, we treat German as the source language from which we translate the reports to other languages.

Original: More speciﬁcally, we follow these steps.
Revised: More speciﬁcally, we follow these steps.

Original: We leverage the Google Translate API 2 to ﬁrst detect the source language of each ECG report.
Revised: We leverage the Google Translate API 2 to ﬁrst detect the source language of each ECG report.

Original: Although the majority of the reports are in German, some are in English, and this language detection step ensures that the ultimate translation is of a higher quality.
Revised: Although the majority of the reports are in German, some are in English, and this language detection step ensures that the ultimate translation is of a higher quality.

Original: We continue to leverage the Google Translate API to translate ECG reports from the identiﬁed source language to the target language of interest.
Revised: We continue to leverage the Google Translate API to translate ECG reports from the identiﬁed source language to the target language of interest.

Original: Due to imperfections in the Google Translate API, certain ECG reports may not be translated in full or translated at all.
Revised: Due to imperfections in the Google Translate API, certain ECG reports may not be translated in full or translated at all.

Original: To minimize the incidence of such cases, we repeat Step 2 several times and stop once we reach the following criterion: we deploy the language detection module of the Google Translate API on the translated reports to conﬁrm that over 90% of them are indeed in the translated language.
Revised: To minimize the incidence of such cases, we repeat Step 2 several times and stop once we reach the following criterion: we deploy the language detection module of the Google Translate API on the translated reports to conﬁrm that over 90% of them are indeed in the translated language.

Original: Although this implies that the ﬁnal translated reports may have some noise, we found that this did not prevent our algorithm from learning appropriately.
Revised: Although this implies that the ﬁnal translated reports may have some noise, we found that this did not prevent our algorithm from learning appropriately.

Original: C I MPLEMENTATION D ETAILS C. N ETWORK A RCHITECTURES
Revised: C I MPLEMENTATION D ETAILS C. N ETWORK A RCHITECTURES

Original: In this section
Revised: In this section

Original: [Table]
Revised: [Table]

Original: we outline the neural network architectures used for our encoder and decoder.
Revised: we outline the neural network architectures used for our encoder and decoder.

Original: More speciﬁcally, we use the architecture shown in Table 4 for the encoder and that shown in Table 5 for the decoder
Revised: More speciﬁcally, we use the architecture shown in Table 4 for the encoder and that shown in Table 5 for the decoder

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: C. E NCODER P RE - TRAINING
Revised: C. E NCODER P RE - TRAINING

Original: In this section, we outline the task used to pre-train the encoder of the captioning system in a supervised manner.
Revised: In this section, we outline the task used to pre-train the encoder of the captioning system in a supervised manner.

Original: Speciﬁcally, we learn an encoder, f θ : u ∈ R P × D → y ∈ R C parameterized by θ , that maps P = 12 D -dimensional ECG signals, u , (where P represents the number of leads) to a C -dimensional output representing the probability assigned to each of the cardiac arrhythmia classes.
Revised: Speciﬁcally, we learn an encoder, f θ : u ∈ R P × D → y ∈ R C parameterized by θ , that maps P = 12 D -dimensional ECG signals, u , (where P represents the number of leads) to a C -dimensional output representing the probability assigned to each of the cardiac arrhythmia classes.

Original: When leveraging the PTB-XL dataset, C = 5 .
Revised: When leveraging the PTB-XL dataset, C = 5 .

Original: For a mini-batch of size, B , and where c i represents the ground-truth class for a particular instance, x i , we learn this behaviour by optimizing the following categorical cross-entropy loss.
Revised: For a mini-batch of size, B , and where c i represents the ground-truth class for a particular instance, x i , we learn this behaviour by optimizing the following categorical cross-entropy loss.

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: We checkpoint, and eventually exploit, the parameters, θ , that coincide with the lowest loss observed on the validation set.
Revised: We checkpoint, and eventually exploit, the parameters, θ , that coincide with the lowest loss observed on the validation set.

Original: This ensures that we use parameters that do not exhibit overﬁtting.
Revised: This ensures that we use parameters that do not exhibit overﬁtting.

Original: C. B ASELINE I MPLEMENTATIONS
Revised: C. B ASELINE I MPLEMENTATIONS

Original: Masked Language Modelling.
Revised: Masked Language Modelling.

Original: Masked language modelling (MLM) can be thought of as analogous to a denoising autoencoder.
Revised: Masked language modelling (MLM) can be thought of as analogous to a denoising autoencoder.

Original: Inputs are perturbed and the network is tasked with generating the original, unperturbed version of the input.
Revised: Inputs are perturbed and the network is tasked with generating the original, unperturbed version of the input.

Original: In the context of natural language processing, a fraction
Revised: In the context of natural language processing, a fraction

Original: F = 0 .
Revised: F = 0 .

Original: 15 of the tokens in a sentence are chosen to be masked.
Revised: 15 of the tokens in a sentence are chosen to be masked.

Original: Of these chosen tokens, 80% are replaced with the token [MASK] , 10% are replaced with a random token from the vocabulary, and the ﬁnal 10% are not replaced at all.
Revised: Of these chosen tokens, 80% are replaced with the token [MASK] , 10% are replaced with a random token from the vocabulary, and the ﬁnal 10% are not replaced at all.

Original: The motivation behind this task lies in the ability of the network to leverage the context of masked tokens to correctly predict them.
Revised: The motivation behind this task lies in the ability of the network to leverage the context of masked tokens to correctly predict them.

Original: This, in turn, allows for the learning of rich representations.
Revised: This, in turn, allows for the learning of rich representations.

Original: In our context, and to allow for a fair comparison to the multilingual pre-training methods, we follow the original implementation introduced by Devlin et al.
Revised: In our context, and to allow for a fair comparison to the multilingual pre-training methods, we follow the original implementation introduced by Devlin et al.

Original: (2019) for each of the language mini-batches.
Revised: (2019) for each of the language mini-batches.

Original: More speciﬁcally, at each iteration, we load N mini-batches corresponding to N languages and perform MLM on each of these batches.
Revised: More speciﬁcally, at each iteration, we load N mini-batches corresponding to N languages and perform MLM on each of these batches.

Original: ELECTRA.
Revised: ELECTRA.

Original: ELECTRA, as opposed to MLM introduced above, is a discriminative language representation learning method.
Revised: ELECTRA, as opposed to MLM introduced above, is a discriminative language representation learning method.

Original: ELECTRA builds upon the implementation of MLM in the following ways.
Revised: ELECTRA builds upon the implementation of MLM in the following ways.

Original: First, instead of masking tokens and tasking the network with generating the original token,
Revised: First, instead of masking tokens and tasking the network with generating the original token,

Original: ELECTRA performs a binary classiﬁcation of whether a token was replaced or not.
Revised: ELECTRA performs a binary classiﬁcation of whether a token was replaced or not.

Original: The motivation for doing so lies in the alleged unnecessary complexity associated with generative language representation learning methods.
Revised: The motivation for doing so lies in the alleged unnecessary complexity associated with generative language representation learning methods.

Original: Moreover, instead of replacing tokens with the [MASK] token,
Revised: Moreover, instead of replacing tokens with the [MASK] token,

Original: ELECTRA proposes to do so by exploiting the predicted outputs of an MLM.
Revised: ELECTRA proposes to do so by exploiting the predicted outputs of an MLM.

Original: This increases the likelihood that replaced tokens are in-distribution.
Revised: This increases the likelihood that replaced tokens are in-distribution.

Original: As a result, ELECTRA simultaneously trains an MLM network and a binary classiﬁer.
Revised: As a result, ELECTRA simultaneously trains an MLM network and a binary classiﬁer.

Original: In our context, we follow the original implementation introduced by Clark et al.
Revised: In our context, we follow the original implementation introduced by Clark et al.

Original: (2020) for each of the N mini-batches.
Revised: (2020) for each of the N mini-batches.

Original: MARGE.
Revised: MARGE.

Original: MARGE is a generative multilingual language representation learning method that exploits source documents in various languages to generate text from a similar yet distinct target document.
Revised: MARGE is a generative multilingual language representation learning method that exploits source documents in various languages to generate text from a similar yet distinct target document.

Original: For example, M source documents with M ∗ S tokens are encoded and leveraged by a decoder to generate the T tokens in the target document.
Revised: For example, M source documents with M ∗ S tokens are encoded and leveraged by a decoder to generate the T tokens in the target document.

Original: In doing so, the network is able to capture relationships between languages and thus learn representations useful for downstream multilingual tasks.
Revised: In doing so, the network is able to capture relationships between languages and thus learn representations useful for downstream multilingual tasks.

Original: In the original implementation (Lewis et al., 2020), similar documents need to be retrieved from a database.
Revised: In the original implementation (Lewis et al., 2020), similar documents need to be retrieved from a database.

Original: In our context, however, our ECG reports are available in N different languages and thus the target document is formed by a report in one language and the source documents are formed by reports in the remaining N − 1 languages.
Revised: In our context, however, our ECG reports are available in N different languages and thus the target document is formed by a report in one language and the source documents are formed by reports in the remaining N − 1 languages.

Original: Since our ECG reports were translated from a single original language, we used reports in this language as target documents.
Revised: Since our ECG reports were translated from a single original language, we used reports in this language as target documents.

Original: For PTB-XL, this amounts to using German.
Revised: For PTB-XL, this amounts to using German.

Original: C. E VALUATION M ETRICS
Revised: C. E VALUATION M ETRICS

Original: As we are mainly interested in the cardiac captioning task, we leverage three automatic metrics commonly used to evaluate image-captioning (BLEU score (Papineni et al., 2002), METEOR (Banerjee & Lavie, 2005) and ROUGE − L
Revised: As we are mainly interested in the cardiac captioning task, we leverage three automatic metrics commonly used to evaluate image-captioning (BLEU score (Papineni et al., 2002), METEOR (Banerjee & Lavie, 2005) and ROUGE − L

Original: (Lin, 2004)).
Revised: (Lin, 2004)).

Original: At a high-level, these metrics quantify the degree of overlap of n-grams between a ground-truth sentence and a generated sentence.
Revised: At a high-level, these metrics quantify the degree of overlap of n-grams between a ground-truth sentence and a generated sentence.

Original: An n-gram can be thought of as a combination of tokens (words) that neighbour one another.
Revised: An n-gram can be thought of as a combination of tokens (words) that neighbour one another.

Original: For example, a 1gram simply consists of all individual tokens in a sentence whereas a 2-gram consists of all pairs of adjacent tokens in the sentence.
Revised: For example, a 1gram simply consists of all individual tokens in a sentence whereas a 2-gram consists of all pairs of adjacent tokens in the sentence.

Original: BLEU Score.
Revised: BLEU Score.

Original: The BLEU score ﬁrst requires calculating the precision of n-grams for a particular value of n.
Revised: The BLEU score ﬁrst requires calculating the precision of n-grams for a particular value of n.

Original: This precision is deﬁned as the number of overlapping n-grams between the generated and ground-truth sentence, Overlap n , divided by the total number of n-grams in the generated sentence Total n .
Revised: This precision is deﬁned as the number of overlapping n-grams between the generated and ground-truth sentence, Overlap n , divided by the total number of n-grams in the generated sentence Total n .

Original: Such a calculation is repeated for multiple values of n ∈ [1 , . . .
Revised: Such a calculation is repeated for multiple values of n ∈ [1 , . . .

Original: , N ] before being averaged and weighted according to a brevity penalty, BP , which penalizes generated sentences which are shorter than the ground-truth sentence.
Revised: , N ] before being averaged and weighted according to a brevity penalty, BP , which penalizes generated sentences which are shorter than the ground-truth sentence.

Original: [Equation]
Revised: [Equation]

Original: METEOR Score.
Revised: METEOR Score.

Original: The METEOR score was designed, for the most part, to explicitly account for recall in n-gram overlap calculations, an operation not included in the BLEU score.
Revised: The METEOR score was designed, for the most part, to explicitly account for recall in n-gram overlap calculations, an operation not included in the BLEU score.

Original: Speciﬁcally, it aligns the ground-truth and generated sentences with one another, calculates the unigram (1- gram) precision and recall, and derives an F-score with greater emphasis on recall than on precision.
Revised: Speciﬁcally, it aligns the ground-truth and generated sentences with one another, calculates the unigram (1- gram) precision and recall, and derives an F-score with greater emphasis on recall than on precision.

Original: Similar to BLEU, it weights the F-score with a sentence brevity penalty, BP .
Revised: Similar to BLEU, it weights the F-score with a sentence brevity penalty, BP .

Original: [Equation]
Revised: [Equation]

Original: ROUGE Score.
Revised: ROUGE Score.

Original: Although there exist multiple ROUGE scores (e.g., ROUGE-N, ROUGE-L,
Revised: Although there exist multiple ROUGE scores (e.g., ROUGE-N, ROUGE-L,

Original: ROUGE-W, and ROUGE-S), we opt for the ROUGE-L score because it obviates the need to set a value of n for the n-grams.
Revised: ROUGE-W, and ROUGE-S), we opt for the ROUGE-L score because it obviates the need to set a value of n for the n-grams.

Original: Formally, ROUGE-L calculates the F-score (geometric mean of precision and recall) of the longest common sub-sequence (LCS) of tokens between the ground-truth sentence with m tokens and the generated sentence with n tokens.
Revised: Formally, ROUGE-L calculates the F-score (geometric mean of precision and recall) of the longest common sub-sequence (LCS) of tokens between the ground-truth sentence with m tokens and the generated sentence with n tokens.

Original: [Equation]
Revised: [Equation]

Revised: C. C LINICAL R EPORT T OKENIZERS

Revised: Throughout the manuscript, we exploit SpaCy as the main natural language processing package.

Revised: Speciﬁcally, we use language-speciﬁc pipelines that are publicly-available such as the following for the Italian language: https://spacy.io/models/it#it_core_news_md .

Revised: Pipelines similar to this allow us to tokenize the reports in speciﬁc languages.

Revised: For all languages, except for

Revised: English, we use the pipelines with the sufﬁx “core news md”.

Revised: For English reports, we use the pipeline with the sufﬁx “core web md”.

Revised: D Q UANTIFICATION OF C LINICAL U TILITY OF R EPORTS

Revised: In this section, we outline the implementation details for conducting the experiments that attempt to demonstrate the clinical utility of reports.

Revised: In order to allay concerns about the quality of the translated ground-truth reports, and to demonstrate the clinical utility of generated reports, we propose to train a model that maps such reports to single-label cardiac abnormalities reﬂected in the corresponding cardiac signals.

Revised: The intuition is that reports which are highly predictive of such abnormalities (which are used for clinical decision-making) are likely to be of clinical value.

Revised: From a mechanistic perspective, we consider the translated ground-truth reports found exclusively in a held-out set ( 3284 reports).

Revised: We split this subset of reports into a training, validation, and test set using a 50 : 20 : 30 split.

Revised: This amounts to 1642 , 656 , and 986 reports, respectively.

Revised: Given the relatively small size of the corpus, and the simplicity of the reports, and to avoid overﬁtting, we opted for a straightforward bag-of-words (BoW) unigram feature representation (maximum features = 50 ).

Revised: We train a Random Forest model on such representations in the training set to perform a multiclass classiﬁcation of cardiac abnormalities.

Revised: These abnormalities are Conduction Disturbance (CD),

Revised: Hypertrophy (HYP), Myocardial Infarction (MI), Normal (NORM), and Ischemic ST-T Changes (STTC).

Revised: We then evaluate the model on the mutually-exclusive set of reports in the test set.

Revised: In order to demonstrate the clinical utility of generated reports, we follow the same procedure as the one described above, with one exception.

Revised: Instead of training on ground-truth reports, we now train on generated reports.

Revised: In all cases, we continue to evaluate on the same set of ground-truth reports in order to allow for a fair comparison between the models in both settings.

Revised: The intuition of this setup is, once again, that reports of higher clinical utility should allow for a model that is more predictive of cardiac abnormalities.