Parsely/streamparse — pull request #47 by kbourgoin (2014-07-28 22:06:54+00:00)
Title: Feature/reorg tests

I'm opening this test for a review of the approach, not because all the tests have been reorganized. If there's support for the ideas in here, then I'll finish it up and replace our current tests with this.

## Overview

Right now, our tests are a bit brittle, the barrier to entry for adding new tests is too high, and debugging failing tests, or gathering code coverage statistics is fairly difficult. This is by no means to say the current tests are *bad*. They work, they test things, and they aren't *too* hard to add new things to. I did this because I want adding new tests to be **stupid easy**. Everything that makes adding new tests easier to write and debug means that more and better tests will get written.

To do this, the big thing I wanted to do was remove the subprocess calls entirely. While they look like Storm, they aren't Storm, and they add a good bit of complication to the setup. For one, it makes using pdb to debug a test impossible, and the break in process space makes code coverage harder to collect.

Finally, we didn't have true integration tests. True integration tests would integrate with Storm and verify that communication was successful. Instead, it integrated with a mock that we created that mimics Storm. Instead of using a subprocess to mock Storm, I'm opting for the mock library to mock individual components. The mock library is a common Python tool for testing that any contributor walking in will know. Using a common tool like that will remove the barrier or learning how our subprocess storm mock works, which makes writing tests that much easier.

*Disclaimer: I've no idea how this will play with python3. Testing that will come later.*

## Benefits

* Easier debugging into running test processes
* Significantly faster tests
* No process creation/teardown for every test
* Simpler testing
* Use common mock tools instead of our own mock framework
* Unit testing means more focused tests covering specific use cases

## Next steps

If other people agree this is a good approach, I'll finish test coverage for the current streamparse version. @msukmanowsky has a large Pull Request coming in, so I'll wait until that's resolved and write all the necessary testing for what's there. Once that's complete, I'll hook up coveralls for streamparse to get code coverage stats. I don't know how useful they'll be, but I'm curious about it and it's integration with Travis-CI.