Abstract-Graphical user interfaces are difficult to test: automated tests are hard to create and maintain, while manual tests are time-consuming, expensive and hard to integrate in a continuous testing process. In this paper, we show that it is possible to crowdsource GUI tests, that is, to outsource them to individuals drawn from a large pool of workers on the Internet, by instantiating virtual machines (VMs) running the system under test and letting testers access the VMs through their web browsers. This enables semi-automated continuous testing of GUIs and usability experiments with large numbers of participants at low cost. Several large experiments on the Amazon Mechanical Turk demonstrate that our approach is technically feasible and sufficiently reliable.
Dolstra et al. REF used virtual machines to run the system under test and enable the crowd workers to accomplish expensive and semi-automatic GUI testing tasks.
Crowdsourcing GUI Tests
2013 IEEE Sixth International Conference on Software Testing, Verification and Validation
2013 IEEE Sixth International Conference on Software Testing, Verification and Validation
Computer Science