TASK DEFINITION: In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.
PROBLEM: In a recent work on sentiment analysis in Turkish BIBREF10, they learn embeddings using Turkish social media. They use the word2vec algorithm, create several unsupervised hand-crafted features, generate document vectors and feed them as input into the support vector machines (SVM) approach. We outperform this baseline approach using more effective word embeddings and supervised hand-crafted features. 
 Question: What baseline method is used?

SOLUTION: using word2vec to create features that are used as input to the SVM

PROBLEM: A meta-research study found only 1 in 9 qualitative papers in Human-Computer Interaction reported inter-rater reliability metrics BIBREF31. Another related area are meta-research and methods papers focused on identifying or preventing low-effort responses from crowdworkers — sometimes called “spam” or “random” responses, or alternatively ”fraudsters” or ”cheaters.” Rates of “self-agreement” are often used, determining if the same person labels the same item differently at a later stage. 
 Question: What are the key issues around whether the gold standard data produced in such an annotation is reliable? 

SOLUTION:  only 1 in 9 qualitative papers in Human-Computer Interaction reported inter-rater reliability metrics low-effort responses from crowdworkers

PROBLEM: As of June 2019, AA had $\sim $50K entries, however, this includes some number of entries that are not truly research publications (for example, forewords, prefaces, table of contents, programs, schedules, indexes, calls for papers/participation, lists of reviewers, lists of tutorial abstracts, invited talks, appendices, session information, obituaries, book reviews, newsletters, lists of proceedings, lifetime achievement awards, erratum, and notes). We discard them for the analyses here. (Note: CL journal includes position papers like squibs, letter to editor, opinion, etc. We do not discard them.) We are then left with 44,896 articles. 
 Question: How many papers are used in experiment?

SOLUTION:
44,896 articles