swirldev/swirl — pull request #68 by WilCrofter (2014-02-06 18:41:54+00:00)
Title: Tests with new (experimental) specification syntax

answerTests2.R implements tests as function calls as discussed at the recent meetup.

The old syntax still works for Data Analysis, Boot Camp, and Open Intro, with the new syntax applying to Intro to R and Test Modules. The AnswerTests column of test modules has been changed to suit the new syntax. The Intro to R modules have not been changed as of this commit, hence won't work in the current state of this branch.

It seems unlikely to us (Gina and Bill) that a fixed suite of test functions will satisfy diverse authors and subject matter, though a small suite may work for a given lesson or course. Since we are essentially unit testing student answers, we think similar considerations apply to swirl as to design of unit testing frameworks. The design adopted by testthat was to provide a few special purpose constructs within the R language so that unit tests needn't be written from scratch every time. Basic constructs, called expectations, are extensible and may be aggregated to form custom tests. The special purpose constructs are few enough to be easily learned. While testthat involves only twelve basic expectations, we have had to write eight functions to cover the five lessons in Intro to R and Test Modules.

We believe something like testthat would a good point of departure for swirl (and similar platforms.) We believe a means of importing customized tests, written by authors for particular lessons, is likely to prove necessary. These could be included in a test.R source file at the lesson or course level, to be imported in the same way as course content.