  Web testing has long been recognized as a notoriously difficult task. Even
nowadays, web testing still heavily relies on manual efforts while automated
web testing is far from achieving human-level performance. Key challenges in
web testing include dynamic content update and deep bugs hiding under
complicated user interactions and specific input values, which can only be
triggered by certain action sequences in the huge search space. In this paper,
we propose WebExplor, an automatic end-to-end web testing framework, to achieve
an adaptive exploration of web applications. WebExplor adopts curiosity-driven
reinforcement learning to generate high-quality action sequences (test cases)
satisfying temporal logical relations. Besides, WebExplor incrementally builds
an automaton during the online testing process, which provides high-level
guidance to further improve the testing efficiency. We have conducted
comprehensive evaluations of WebExplor on six real-world projects, a commercial
SaaS web application, and performed an in-the-wild study of the top 50 web
applications in the world. The results demonstrate that in most cases WebExplor
can achieve a significantly higher failure detection rate, code coverage, and
efficiency than existing state-of-the-art web testing techniques. WebExplor
also detected 12 previously unknown failures in the commercial web application,
which have been confirmed and fixed by the developers. Furthermore, our
in-the-wild study further uncovered 3,466 exceptions and errors.
