Question: What are the key issues around whether the gold standard data produced in such an annotation is reliable? 
Answer:  only 1 in 9 qualitative papers in Human-Computer Interaction reported inter-rater reliability metrics, low-effort responses from crowdworkers