exercism/exercism.io — pull request #2087 by bluerogue251 (2014-12-02 22:26:38+00:00)
Title: WIP Switch to Postgres for tests

* Allow case insensitive user queries as requested in #1805.
* Switch test database to Postgres; update README and tests

There are still two issues:

1. This test still fails sometimes: `NitpickAppTest#test_language_when_and_submission_present`. All the other tests I adjusted to not rely on implicit ordering of records pulled out of the db. But this test file, I can't see where the implicit ordering is coming from. Any help appreciated!

2. The test performance degradation. As @SleeplessByte suggested, I logged long-running queries but didn't find any solid evidence (only 3 queries over 10 ms for the whole test suite, none over 30ms). I haven't made any progress on figuring out what's causing it.

I think there is a major gain in safety from testing on the same database as is used in production. But I think the performance hit is a big deal, too.

It's definitely OK if we don't end up merging this due to the test performance. What are everyone's thoughts?