Test-Retest Reliability Used to assess the consistency of a measure from one time to another. Parallel-Forms Reliability Used to assess the consistency of the results of two tests constructed in the same way from the same content domain. Internal Consistency Reliability Used to assess the consistency of results across items within a test. Let's discuss each of these in turn. Whenever you use humans as a part of your measurement procedure, you have to worry about whether the results you get are reliable or consistent.
People are notorious for their inconsistency.
We are easily distractible. We get tired of doing repetitive tasks. You probably should establish inter-rater reliability outside of the context of the measurement in your study. After all, if you use data from your study to establish reliability, and you find that reliability is low, you're kind of stuck. Probably it's best to do this as a side study or pilot study.