norman/telescope — pull request #19 by Odie (2012-08-13 18:36:05+00:00)
Title: Sandboxing individual tests

Hi there...

I believe my last pull request actually introduced a bug (Which I will remedy here!). The last pull request placed load_context into its own sandbox. It appears that any of the functions/blocks that it loaded also inherited these new environments by default.

This appears to be okay, until we hit the run() function and all the test blocks gets their environment changed to the global environment. The after and before blocks will still run with the sandbox that was given to them during load_context(). So even though the before and after blocks are being run, they are altering an environment that's effectively invisible to the test blocks.

The pull request contains the following changes:
* Proper sandboxing

During run(), we setup a new environment to be shared between all before/after/test blocks on a per-test basis. That means each test will start with a clean-slate. No test can accidentally leave behind a global variable that may change the result of the test in unforeseen ways.

* Proper teardown order

The setup code (before blocks) has always been run in the correct order, proceeding from the outer-most into the inner-most blocks. However, it also runs the tear-down code in the same order, meaning the teardown code for the outer-most blocks is run first before the inner blocks have an opportunity to run.

This *may* cause problems if the nested blocks actually created nested data structures. When the teardown code is run for the outer-most contexts, it will likely remove any outer-most data structures also, making any inner context teardown code somewhat pointless. It's probably only problematic if there are any system level resources that should be shutdown explicitly.