qinchuanhui/UDA-QA huggingface dataset:
Question: Which dataset do they use?

Evidence:
raw_evidence: In this study, we investigate the performance of our proposed models on WSJ BIBREF5 . We used the same definitions of the training, development and test set as the Kaldi s5 recipe BIBREF18 . The raw speech waveforms were segmented into multiple frames with a 25ms window size and a 10ms step size. We normalized the raw speech waveform into the range -1 to 1. For spectral based features such as MFCC and log Mel-spectrogram, we normalized the features for each dimension into zero mean and unit variance. For WSJ, we separated into two experiments by using WSJ-SI84 only and WSJ-SI284 data. We used dev_93 for our validation set and eval_92 for our test set. We used the character sequence as our decoder target and followed the preprocessing step proposed by BIBREF19 . The text from all the utterances was mapped into a 32-character set: 26 (a-z) alphabet, apostrophe, period, dash, space, noise, and “eos".
raw_evidence: An example of our transfer learning results is shown in Figure FIGREF8 , and Table TABREF14 shows the speech recognition performance in CER for both the WSJ-SI84 and WSJ-SI284 datasets. We compared our method with several published models like CTC, Attention Encoder-Decoder and Joint CTC-Attention model that utilize CTC for training the encoder part. Besides, we also train our own baseline Attention Encoder-Decoder with Mel-scale spectrogram. The difference between our Attention Encoder-Decoder (“Att Enc-Dec (ours)", “Att Enc-Dec Wav2Text") with Attention Encoder-Decoder from BIBREF24 (“Att Enc-Dec Content", “Att Enc-Dec Location") is we used the current hidden states to generate the attention vector instead of the previous hidden states. Another addition is we utilized “input feedback" method BIBREF13 by concatenating the previous context vector into the current input along with the character embedding vector. By using those modifications, we are able to improve the baseline performance.

Answers:
WSJ
WSJ-SI84, WSJ-SI284
Question: What model is used to encode the images?

Evidence:
raw_evidence: Our model extends the image description model by BIBREF0 , which consists of an encoder-decoder architecture. The encoder is a Convolutional Neural Network (CNN) and the decoder is a Long Short-Term Memory (LSTM) network, as presented in Figure 2 . The image is passed through the encoder generating the image representation that is used by the decoder to know the content of the image and generate the description word by word. In the following, we describe how we extended this model for the visual storytelling task.
raw_evidence: The model's first component is a Recurrent Neural Network (RNN), more precisely an LSTM that summarizes the sequence of images. At every timestep $t$ the network takes as input an image $I_i$ where $i\in \lbrace 1,2,3,4,5\rbrace $ from the sequence. At time $t=5$ , the LSTM has encoded the 5 images and provides the sequence's context through its last hidden state denoted by $h_e^{(t)}$ . The representation of the images was obtained through Inception V3.

Answers:
a Convolutional Neural Network (CNN)
LSTM
Question: How is the sequential nature of the story captured?

Evidence:
raw_evidence: The decoder is the second LSTM network that uses the information obtained from the encoder to generate the sequence's story. The first input $x_0$ to the decoder is the image for which the text is being generated. The last hidden state from the encoder $h_e^{(t)}$ is used to initialize the first hidden state of the decoder $h_d^{(0)}$ . With this strategy, we provide the decoder with the context of the whole sequence and the content of the current image (i.e. global and local information) to generate the corresponding text that will contribute to the overall story.
raw_evidence: Our proposed architecture is presented in Figure 3 . For each image in the sequence, we obtain its representation $\lbrace e(I_1),...,e(I_5)\rbrace $ using Inception v3. The encoder takes the images in order, one at every timestep $t$ . At time $t=5$ , we obtain the context vector through $h_e^{(t)}$ (represented by $\mathbf {Z}$ ). This vector is used to initialize each decoder's hidden state while the first input to each decoder is its corresponding image embedding $e(I_i)$ . Each decoder generates a sequence of words $\lbrace p_1,...,p_{n}\rbrace $ for each image in the sequence. The final story is the concatenation of the output of the 5 decoders.

Answers:
we provide the decoder with the context of the whole sequence and the content of the current image (i.e. global and local information) to generate the corresponding text that will contribute to the overall story
The encoder takes the images in order, one at every timestep $t$ . At time $t=5$ , we obtain the context vector through $h_e^{(t)}$ (represented by $\mathbf {Z}$ ). This vector is used to initialize each decoder's hidden state while the first input to each decoder is its corresponding image embedding $e(I_i)$ . Each decoder generates a sequence of words $\lbrace p_1,...,p_{n}\rbrace $ for each image in the sequence.
Question: Is the position in the sequence part of the input?

Evidence:
raw_evidence: The model's first component is a Recurrent Neural Network (RNN), more precisely an LSTM that summarizes the sequence of images. At every timestep $t$ the network takes as input an image $I_i$ where $i\in \lbrace 1,2,3,4,5\rbrace $ from the sequence. At time $t=5$ , the LSTM has encoded the 5 images and provides the sequence's context through its last hidden state denoted by $h_e^{(t)}$ . The representation of the images was obtained through Inception V3.

Answers:
No
Yes
Question: Do the decoder LSTMs all have the same weights?

Evidence:
raw_evidence: Our model contains five independent decoders, one for each image in the sequence. All the 5 decoders use the last hidden state of the encoder (i.e. the context) as its first hidden state and take the corresponding image embedding as its first input. In this way, the first decoder generates the sequence of words for the first image in the sequence, the second decoder for the second image in the sequence, and so on. This allows each decoder to learn a specific language model for each position of the sequence. For instance, the first decoder will learn the opening sentences of the story while the last decoder the closing sentences. The word embeddings were computed using word2vec BIBREF8 .
raw_evidence: Our model contains five independent decoders, one for each image in the sequence. All the 5 decoders use the last hidden state of the encoder (i.e. the context) as its first hidden state and take the corresponding image embedding as its first input. In this way, the first decoder generates the sequence of words for the first image in the sequence, the second decoder for the second image in the sequence, and so on. This allows each decoder to learn a specific language model for each position of the sequence. For instance, the first decoder will learn the opening sentences of the story while the last decoder the closing sentences. The word embeddings were computed using word2vec BIBREF8 .

Answers:
No
No
Question: Is fine-tuning required to incorporate these embeddings into existing models?

Evidence:
raw_evidence: While it is obvious that our embeddings can be used as features for new predictive models, it is also very easy to incorporate our learned Dolores embeddings into existing predictive models on knowledge graphs. The only requirement is that the model accepts as input, an embedding layer (for entities and relations). If a model fulfills this requirement (which a large number of neural models on knowledge graphs do), we can just use Dolores embeddings as a drop-in replacement. We just initialize the corresponding embedding layer with Dolores embeddings. In our evaluation below, we show how to improve several state-of-the-art models on various tasks simply by incorporating Dolores as a drop-in replacement to the original embedding layer.
raw_evidence: While it is obvious that our embeddings can be used as features for new predictive models, it is also very easy to incorporate our learned Dolores embeddings into existing predictive models on knowledge graphs. The only requirement is that the model accepts as input, an embedding layer (for entities and relations). If a model fulfills this requirement (which a large number of neural models on knowledge graphs do), we can just use Dolores embeddings as a drop-in replacement. We just initialize the corresponding embedding layer with Dolores embeddings. In our evaluation below, we show how to improve several state-of-the-art models on various tasks simply by incorporating Dolores as a drop-in replacement to the original embedding layer.

Answers:
No
No
Question: How are meaningful chains in the graph selected?

Evidence:
raw_evidence: While it is obvious that our embeddings can be used as features for new predictive models, it is also very easy to incorporate our learned Dolores embeddings into existing predictive models on knowledge graphs. The only requirement is that the model accepts as input, an embedding layer (for entities and relations). If a model fulfills this requirement (which a large number of neural models on knowledge graphs do), we can just use Dolores embeddings as a drop-in replacement. We just initialize the corresponding embedding layer with Dolores embeddings. In our evaluation below, we show how to improve several state-of-the-art models on various tasks simply by incorporating Dolores as a drop-in replacement to the original embedding layer.
raw_evidence: Having generated a set of paths on knowledge graphs representing local contexts of entities and relations, we are now ready to utilize the machinery of language modeling using deep neural networks to learn Dolores embeddings.
After having estimated the parameters of the Dolores learner, we now extract the context-independent and context-dependent representations for each entity and relation and combine them to obtain Dolores embeddings. More specifically, Dolores embeddings are task specific combination of the context-dependent and context-independent representations learned by our learner. Note that our learner (which is an $L$ -layer Bi-Directional LSTM) computes a set of $2L + 1$ representations for each entity-relation pair which we denote by: $ R_t = [ x_t, \overrightarrow{h_{t,i}}, \overleftarrow{h_{t,i}} \mid i = 1, 2, \cdots , \textit {L} ], $

Answers:
No
utilize the machinery of language modeling using deep neural networks to learn Dolores embeddings.
Question: Do they remove seasonality from the time series?

Evidence:
raw_evidence: Data Pre-Processing
Stock prices data collected is not complete understandably because of weekends and public holidays when the stock market does not function. The missing data is approximated using a simple technique by Goel BIBREF17 . Stock data usually follows a concave function. So, if the stock value on a day is x and the next value present is y with some missing in between. The first missing value is approximated to be (y+x)/2 and the same method is followed to fill all the gaps.

Answers:
No
Question: What is the dimension of the embeddings?

Evidence:
raw_evidence: Word2vec representation is far better, advanced and a recent technique which functions by mapping words to a 300 dimensional vector representations. Once every word of the language has been mapped to a unique vector, vectors of words can be summed up yielding a resultant vector for any given collection of words BIBREF19 . Relationship between the words is exactly retained in this form of representation. Word vectors difference between Rome and Italy is very close to the difference between vectors of France and Paris This sustained relationship between word concepts makes word2vec model very attractive for textual analysis. In this representation, resultant vector which is sum of 300 dimensional vectors of all words in a tweet acts as features to the model.
raw_evidence: Word2vec representation is far better, advanced and a recent technique which functions by mapping words to a 300 dimensional vector representations. Once every word of the language has been mapped to a unique vector, vectors of words can be summed up yielding a resultant vector for any given collection of words BIBREF19 . Relationship between the words is exactly retained in this form of representation. Word vectors difference between Rome and Italy is very close to the difference between vectors of France and Paris This sustained relationship between word concepts makes word2vec model very attractive for textual analysis. In this representation, resultant vector which is sum of 300 dimensional vectors of all words in a tweet acts as features to the model.

Answers:
300
300
Question: What dataset is used to train the model?

Evidence:
raw_evidence: A total of 2,50,000 tweets over a period of August 31st, 2015 to August 25th,2016 on Microsoft are extracted from twitter API BIBREF15 . Twitter4J is a java application which helps us to extract tweets from twitter. The tweets were collected using Twitter API and filtered using keywords like $ MSFT, # Microsoft, #Windows etc. Not only the opinion of public about the company's stock but also the opinions about products and services offered by the company would have a significant impact and are worth studying. Based on this principle, the keywords used for filtering are devised with extensive care and tweets are extracted in such a way that they represent the exact emotions of public about Microsoft over a period of time. The news on twitter about Microsoft and tweets regarding the product releases were also included. Stock opening and closing prices of Microsoft from August 31st, 2015 to August 25th, 2016 are obtained from Yahoo! Finance BIBREF16 .
raw_evidence: A total of 2,50,000 tweets over a period of August 31st, 2015 to August 25th,2016 on Microsoft are extracted from twitter API BIBREF15 . Twitter4J is a java application which helps us to extract tweets from twitter. The tweets were collected using Twitter API and filtered using keywords like $ MSFT, # Microsoft, #Windows etc. Not only the opinion of public about the company's stock but also the opinions about products and services offered by the company would have a significant impact and are worth studying. Based on this principle, the keywords used for filtering are devised with extensive care and tweets are extracted in such a way that they represent the exact emotions of public about Microsoft over a period of time. The news on twitter about Microsoft and tweets regarding the product releases were also included. Stock opening and closing prices of Microsoft from August 31st, 2015 to August 25th, 2016 are obtained from Yahoo! Finance BIBREF16 .

Answers:
2,50,000 tweets, Stock opening and closing prices of Microsoft from August 31st, 2015 to August 25th, 2016
Collected tweets and opening and closing stock prices of Microsoft.
Question: What is the previous state of the art?

Evidence:
raw_evidence: Looking at the current WinoGrande leaderboard, it appears that the previous state of the art is based on RoBERTa BIBREF2, which can be characterized as an encoder-only transformer architecture. Since T5-3B is larger than RoBERTa, it cannot be ruled out that model size alone explains the performance gain. However, when coupled with the observations of Nogueira et al. BIBREF7, T5's “generative capability”, i.e., its ability to generate fluent text, honed through pretraining, seems to play an important role. The fact that the choice of target tokens affects prediction accuracy is consistent with this observation. How and why is the subject of ongoing work.
raw_evidence: Looking at the current WinoGrande leaderboard, it appears that the previous state of the art is based on RoBERTa BIBREF2, which can be characterized as an encoder-only transformer architecture. Since T5-3B is larger than RoBERTa, it cannot be ruled out that model size alone explains the performance gain. However, when coupled with the observations of Nogueira et al. BIBREF7, T5's “generative capability”, i.e., its ability to generate fluent text, honed through pretraining, seems to play an important role. The fact that the choice of target tokens affects prediction accuracy is consistent with this observation. How and why is the subject of ongoing work.

Answers:
RoBERTa
RoBERTa
Question: What criteria are used to select the 8,640 English sentences?

Evidence:
raw_evidence: We decided to use sentences involving at least one race- or gender-associated word. The sentences were intended to be short and grammatically simple. We also wanted some sentences to include expressions of sentiment and emotion, since the goal is to test sentiment and emotion systems. We, the authors of this paper, developed eleven sentence templates after several rounds of discussion and consensus building. They are shown in Table TABREF3 . The templates are divided into two groups. The first type (templates 1–7) includes emotion words. The purpose of this set is to have sentences expressing emotions. The second type (templates 8–11) does not include any emotion words. The purpose of this set is to have non-emotional (neutral) sentences.
raw_evidence: We generated sentences from the templates by replacing INLINEFORM0 person INLINEFORM1 and INLINEFORM2 emotion word INLINEFORM3 variables with the values they can take. In total, 8,640 sentences were generated with the various combinations of INLINEFORM4 person INLINEFORM5 and INLINEFORM6 emotion word INLINEFORM7 values across the eleven templates. We manually examined the sentences to make sure they were grammatically well-formed. Notably, one can derive pairs of sentences from the EEC such that they differ only in one word corresponding to gender or race (e.g., `My daughter feels devastated' and `My son feels devastated'). We refer to the full set of 8,640 sentences as Equity Evaluation Corpus.

Answers:
Sentences involving at least one race- or gender-associated word,  sentence  have to be short and grammatically simple,  sentence have to  include expressions of sentiment and emotion.
generated with the various combinations of INLINEFORM4 person INLINEFORM5 and INLINEFORM6 emotion word INLINEFORM7 values across the eleven templates, differ only in one word corresponding to gender or race
Question: Is collected multimodal in cabin dataset public?

Evidence:
raw_evidence: We explored leveraging multimodality for the NLU module in the SDS pipeline. As our AMIE in-cabin dataset has video and audio recordings, we investigated 3 modalities for the NLU: text, audio, and video. For text (language) modality, our previous work BIBREF1 presents the details of our best-performing Hierarchical & Joint Bi-LSTM models BIBREF3, BIBREF4, BIBREF5, BIBREF6 (H-Joint-2, see SECREF5) and the results for utterance-level intent recognition and word-level slot filling via transcribed and recognized (ASR output) textual data, using word embeddings (GloVe BIBREF7) as features. This study explores the following multimodal features:

Answers:
No
Question: Is the model tested against any baseline?

Evidence:
raw_evidence: For each domain text data, we train an n-gram language modelBIBREF1 that is optimized for that domain. As the results, we have more than 10 language models. These language models are combined based on perplexity calculated on a small text of a domain that we want to optimize for.
To further improve the performance, we adopt system combination on the decoding lattice level. By combining systems, we can take advantage of the strength of each model that is optimized for different domains. The results for 2 test sets is showed on Table TABREF17 and TABREF18.

Answers:
No
No
Question: What is the language model combination technique used in the paper?

Evidence:
raw_evidence: To further improve the performance, we adopt system combination on the decoding lattice level. By combining systems, we can take advantage of the strength of each model that is optimized for different domains. The results for 2 test sets is showed on Table TABREF17 and TABREF18.
As we can see, for both test sets, system combination significantly reduce the WER. The best result for vlsp2018 of 4.85% WER is obtained by the combination weights 0.6:0.4 where 0.6 is given to the general language model and 0.4 is given to the conversation one. On the vlsp2019 set, the ratio is change slightly by 0.7:0.3 to deliver the best result of 15.09%.
raw_evidence: To further improve the performance, we adopt system combination on the decoding lattice level. By combining systems, we can take advantage of the strength of each model that is optimized for different domains. The results for 2 test sets is showed on Table TABREF17 and TABREF18.

Answers:
system combination on the decoding lattice level, combination weights
system combination on the decoding lattice level
Question: What are the deep learning architectures used in the task?

Evidence:
raw_evidence: We adopt a DNN-based acoustic model BIBREF0 with 11 hidden layers and the alignment used to train the model is derived from a HMM-GMM model trained with SAT criterion. In a conventional Gaussian Mixture Model - Hidden Markov Model (GMM-HMM) acoustic model, the state emission log-likelihood of the observation feature vector $o_t$ for certain tied state $s_j$ of HMMs at time $t$ is computed as

Answers:
DNN-based acoustic model BIBREF0
Question: Do they train their model starting from a checkpoint?

Evidence:
raw_evidence: This section gives detailed experiment settings. The method is evaluated on the BERTbase model, which has 12 layers, 12 self-attention heads with a hidden size of 768.
To accelerate the training speed, two-phase training BIBREF1 is adopted. The first phase uses a maximal sentence length of 128, and 512 for the second phase. The numbers of training steps of two phases are 50K and 40K for the BERTBase model. We used AdamW BIBREF13 optimizer with a learning rate of 1e-4, a $\beta _1$ of 0.9, a $\beta _2$ of 0.999 and a L2 weight decay rate of $0.01$. The first 10% of the total steps are used for learning rate warming up, followed by the linear decay schema. We used a dropout probability of 0.1 on all layers. The data used for pre-training is the same as BERT, i.e., English Wikipedia (2500M words) and BookCorpus (800M words) BIBREF14. For the Masked LM task, we followed the same masking rate and settings as in BERT.
raw_evidence: This section gives detailed experiment settings. The method is evaluated on the BERTbase model, which has 12 layers, 12 self-attention heads with a hidden size of 768.
To accelerate the training speed, two-phase training BIBREF1 is adopted. The first phase uses a maximal sentence length of 128, and 512 for the second phase. The numbers of training steps of two phases are 50K and 40K for the BERTBase model. We used AdamW BIBREF13 optimizer with a learning rate of 1e-4, a $\beta _1$ of 0.9, a $\beta _2$ of 0.999 and a L2 weight decay rate of $0.01$. The first 10% of the total steps are used for learning rate warming up, followed by the linear decay schema. We used a dropout probability of 0.1 on all layers. The data used for pre-training is the same as BERT, i.e., English Wikipedia (2500M words) and BookCorpus (800M words) BIBREF14. For the Masked LM task, we followed the same masking rate and settings as in BERT.

Answers:
No
No
Question: What BERT model do they test?

Evidence:
raw_evidence: This section gives detailed experiment settings. The method is evaluated on the BERTbase model, which has 12 layers, 12 self-attention heads with a hidden size of 768.
raw_evidence: This section gives detailed experiment settings. The method is evaluated on the BERTbase model, which has 12 layers, 12 self-attention heads with a hidden size of 768.

Answers:
BERTbase
BERTbase
Question: What downstream tasks are evaluated?

Evidence:
raw_evidence: We evaluate the effect batch size has on active learning stopping methods for text classification. We use the publicly available 20Newsgroups dataset in our experiments.

Answers:
text classification