FoundationDB/sql-layer-odbc — pull request #3 by louisrli (2014-01-07 21:41:56+00:00)
Title: Replace pg regress

The psqlODBC driver uses `pg_regress` to run its tests. However, `pg_regress` is closely tied to the Postgres ecosystem (and its syntax).

Luckily, the tests are self-contained executables. This pull requests adds a `test_runner.py`, which at the moment, is a one-off Python script that runs the tests and compares the outputs just as `pg_regress` did.

The output looks something like (note that five tests are now passing, but I do need to confirm that they are not false positives):

```
alter-test | FAILED (nonzero exit code)
arraybinding-test | FAILED (nonzero exit code)
boolsaschar-test | OK
catalogfunctions-test | FAILED (nonzero exit code)
commands-test | FAILED (nonzero exit code)
connect-test | OK
cursors-test | FAILED (nonzero exit code)
cvtnulldate-test | FAILED (nonzero exit code)
dataatexecution-test | OK
getresult-test | FAILED (nonzero exit code)
insertreturning-test | FAILED (nonzero exit code)
multistmt-test | OK
notice-test | FAILED (nonzero exit code)
params-test | FAILED (mismatched output)
positioned-update-test | FAILED (nonzero exit code)
prepare-test | FAILED (nonzero exit code)
quotes-test | FAILED (nonzero exit code)
select-test | OK
stmthandles-test | FAILED (nonzero exit code)
```

The other reason something can fail is `mismatched output`

`run_tests.sh` also sets up the sample tables that `pg_regress` was setting up.

Results are also written to disk in `results/` (individual test outputs) and `results.log` (the table above)