They then compare and discuss their ratings. Although it often takes an hour or more to get through one call, this is perhaps the best single method to gain consistency among raters. By making this comparison, raters can see whether they are more lenient or more stringent than the team. Subsequent calibration sessions should focus on clarifying areas of inconsistency. The fifth component of effective monitoring reflects the extent to which monitoring scores actually correspond to customer evaluations of how they've been treated.
Validity needs to be built in to the design of the monitoring process.
The form must be comprehensive, so that it includes all major elements of customer treatment. Draw on customer focus group input and verbatims on customer surveys to identify those CSR behaviors which are particularly annoying or satisfying. The standards which are applied must reflect the customer's viewpoint, so that behaviors which delight the customer are rated highly and behaviors which the customer found unsatisfactory are rated low. To measure the validity of a monitoring program, the customers whose calls have been monitored are then surveyed. The survey must focus the customer on the specific behaviors demonstrated by the CSR on the specific call.