## 2 Material and Methods
### Design  
A sequential, explanatory mixed-method research approach was employed to elucidate the technology use for an effective implementation of the ODK Scan paper-to-digital system. However, the mixed method research approach involves combining the quantitative analysis with qualitative reasoning. This research approach includes application of three methods: (1) quantitative assessment of the data quality at both app and user levels; (2) quantitative measurement of the time spent in filling forms, data transfer, aggregation and verification; and (3) qualitative feedback from the field workers (users) to capture their complete experience. However, each of these methods is explored in more detail below.  
(1) Quantitative assessment of the data quality was done at both the app and user levels. At the application level, accuracy of ODK Scan's ability to recognize fill-in bubbles and number boxes was measured by comparing the raw Scan values (the digitized values which Scan's algorithm produced before human verification) with the data appearing in image snippets of the corresponding data fields. To calculate this, the team hired a Seattle USA-based data analyst to examine each individual number and fill-in bubble image snippet, to record the observed \"ground truth\" value as observed from the image, and to compare it with the corresponding raw value that ODK Scan processing recorded. Comparison of these field value pairs allowed us to calculate ODK Scan's accuracy rates for all the number and fill-in bubble fields.  
At the user-level, the workflow allowed users to improve the quality of the data through verification and editing data fields manually. User-level data quality was assessed by comparing the same \"ground truth\" values for each field to the final values that the field worker synced with the server. These accuracy rates were predicted to be higher than the app-level accuracy rate because the field workers had the opportunity to correct errors from the \"raw\" Scan processing.  
Furthermore, we calculated the accuracy rates of the existing (non-ODK Scan) manual entry processes by comparing the field values observed from the hardcopy TB03 register pages to the values manually entered by the data entry worker in the Microsoft Excel datasheets. This would provide us with a baseline for manual entry accuracy rate. Accuracy rates for both user-level digitization processes were calculated based on the full field value, rather than by individual digit or bubble (binary assessment of correct or incorrect for each field).  
(2) Quantitative measurements of the time spent in filling forms, data transfer, aggregation and verification were recorded in the routine data management process using direct observation. They were then compared with the endline measures for the same data management processes using the ODK Scan system.  
(3) Gathering of qualitative feedback on app use experience was done through surveys and interviews. The survey captured Likert-scale assessment, while face-to-face interviews using a semi-structured questionnaire were carried out with each of the field workers. Their experiences with app usability and workflow modification were explored to assess the effectiveness of the ODK Scan paper-to-digital system from a usability perspective.