instruction:
In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.
question:
First preference is given to the labels that are perfectly matching in all the neural annotators. In Table TABREF11, we can see that both datasets have about 40% of exactly matching labels over all models (AM). Then priority is given to the context-based models to check if the label in all context models is matching perfectly. In case two out of three context models are correct, then it is being checked if that label is also produced by at least one of the non-context models. Then, we allow labels to rely on these at least two context models. As a result, about 47% of the labels are taken based on the context models (CM). When we see that none of the context models is producing the same results, then we rank the labels with their respective confidence values produced as a probability distribution using the $softmax$ function. The labels are sorted in descending order according to confidence values. Then we check if the first three (case when one context model and both non-context models produce the same label) or at least two labels are matching, then we allow to pick that one. There are about 3% in IEMOCAP and 5% in MELD (BM).

Finally, when none the above conditions are fulfilled, we leave out the label with an unknown category. This unknown category of the dialogue act is labeled with `xx' in the final annotations, and they are about 7% in IEMOCAP and 11% in MELD (NM). 
 Question: How does the ensemble annotator extract the final label?
answer:
First preference is given to the labels that are perfectly matching in all the neural annotators. In case two out of three context models are correct, then it is being checked if that label is also produced by at least one of the non-context models. When we see that none of the context models is producing the same results, then we rank the labels with their respective confidence values produced as a probability distribution using the $softmax$ function. The labels are sorted in descending order according to confidence values. Then we check if the first three (case when one context model and both non-context models produce the same label) or at least two labels are matching, then we allow to pick that one.  Finally, when none the above conditions are fulfilled, we leave out the label with an unknown category.


question:
Our target data set consists of a corpus of discharge summaries, admission notes, individual encounter notes, and other clinical notes from 220 patients in the OnTrackTM program at McLean Hospital. OnTrackTM is an outpatient program, focusing on treating adults ages 18 to 30 who are experiencing their first episodes of psychosis. The length of time in the program varies depending on patient improvement and insurance coverage, with an average of two to three years. The program focuses primarily on early intervention via individual therapy, group therapy, medication evaluation, and medication management. See Table TABREF2 for a demographic breakdown of the 220 patients, for which we have so far extracted approximately 240,000 total EHR paragraphs spanning from 2011 to 2014 using Meditech, the software employed by McLean for storing and organizing EHR data.

These patients are part of a larger research cohort of approximately 1,800 psychosis patients, which will allow us to connect the results of this EHR study with other ongoing research studies incorporating genetic, cognitive, neurobiological, and functional outcome data from this cohort.

We also use an additional data set for training our vector space model, comprised of EHR texts queried from the Research Patient Data Registry (RPDR), a centralized regional data repository of clinical data from all institutions in the Partners HealthCare network. These records are highly comparable in style and vocabulary to our target data set. The corpus consists of discharge summaries, encounter notes, and visit notes from approximately 30,000 patients admitted to the system's hospitals with psychiatric diagnoses and symptoms. This breadth of data captures a wide range of clinical narratives, creating a comprehensive foundation for topic extraction. 
 Question: What datasets did the authors use?
answer:
 a corpus of discharge summaries, admission notes, individual encounter notes, and other clinical notes from 220 patients in the OnTrackTM program at McLean Hospital an additional data set for training our vector space model, comprised of EHR texts queried from the Research Patient Data Registry (RPDR)


question:
The GENIA Corpus BIBREF3 contains biomedical abstracts from the PubMed database. We use GENIA technical term annotations 3.02, which cover linguistic expressions to entities of interest in molecular biology, e.g. proteins, genes and cells. CoNLL2003 BIBREF14 is a standard NER dataset based on the Reuters RCV-1 news corpus. It covers named entities of type person, location, organization and misc.

For testing the overall annotation performance, we utilize CoNLL2003-testA and a 50 document split from GENIA.  
 Question: what standard dataset were used?
answer:

The GENIA Corpus  CoNLL2003