TASK DEFINITION: In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
PROBLEM: Probing Methodology and Modeling ::: Task Definition and Modeling ::: Baselines and Sanity Checks.
When creating synthetic datasets, it is important to ensure that systematic biases, or annotation artifacts BIBREF41, are not introduced into the resulting probes and that the target datasets are sufficiently challenging (or good, in the sense of BIBREF42). To test for this, we use several of the MCQA baseline models first introduced in BIBREF0, which take inspiration from the LSTM-based models used in BIBREF43 for NLI and various partial-input baselines based on these models.
Question: How do they control for annotation artificats?

SOLUTION: we use several of the MCQA baseline models first introduced in BIBREF0

PROBLEM: Suppose there are $M$ total layers, and define a subset of these layers at which to apply the loss function: $K = \lbrace k_1, k_2, ..., k_L
brace \subseteq \lbrace 1,..,M-1
brace $. The total objective function is then defined as

where $Z_{k_l}$ is the $k_l$-th Transformer layer activations, $Y$ is the ground-truth transcription for CTC and context dependent states for hybrid ASR, and $Loss(P, Y)$ can be defined as CTC objective BIBREF11 or cross entropy for hybrid ASR. T
Question: Do they just sum up all the loses the calculate to end up with one single loss?

SOLUTION: No

PROBLEM: We use the Ultrax Typically Developing dataset (UXTD) from the publicly available UltraSuite repository BIBREF19 . This dataset contains synchronized acoustic and ultrasound data from 58 typically developing children, aged 5-12 years old (31 female, 27 male). The data was aligned at the phone-level, according to the methods described in BIBREF19 , BIBREF25 . The data was recorded using an Ultrasonix SonixRP machine using Articulate Assistant Advanced (AAA) software at INLINEFORM0 121fps with a 135 field of view. A single ultrasound frame consists of 412 echo returns from each of the 63 scan lines (63x412 raw frames).
Question: What are the characteristics of the dataset?

SOLUTION:
synchronized acoustic and ultrasound data from 58 typically developing children, aged 5-12 years old (31 female, 27 male) data was aligned at the phone-level 121fps with a 135 field of view single ultrasound frame consists of 412 echo returns from each of the 63 scan lines (63x412 raw frames)