maage/test-nginx — pull request #1 by maage (2014-04-12 13:15:19+00:00)
Title: new feature: must_exit

Feature to test if exit value is something else than 0. Used to test
config parser can notice bogus config. If this is set, no requests
are done.

This is for #10, but I'm not quite sure if this is enough.

Valgrind / stap / profiling etc. are not supported. I think there is no point of running under those when you are expected to exit program readily. Maybe there should be additional checks for those. For now you can guard at test file instead, example below. And so every test should have must_exit or none should.

Quick test for echo module to demonstrate. Test 1 uses undocumented flag and it is not noticed. Test 2 has bogus config and it is noticed as it should.

```
% cat t/echo-bogus.t
# vi:filetype=

use lib 'lib';
use Test::Nginx::Socket;

if ($ENV{TEST_NGINX_USE_VALGRIND} || $ENV{TEST_NGINX_CHECK_LEAK} || $ENV{TEST_NGINX_USE_STAP}) {
plan skip_all => 'must_exit is not compatible with extra checks';
}

repeat_each(1);

plan tests => repeat_each() * blocks();

#$Test::Nginx::LWP::LogLevel = 'debug';

run_tests();

__DATA__

=== TEST 1: echo unknown flag
--- config
location /echo {
echo -x hello;
}
--- must_exit: 1
--- request
GET /echo

=== TEST 2: echo_sleep bad value
--- config
location /echo {
#echo_sleep -1;
echo_sleep;
echo hello;
}
--- must_exit: 1
--- request
GET /echo
```

Test run:

```
TEST_NGINX_BINARY=nginx/objs/nginx prove -l -r t/echo-bogus.t
t/echo-bogus.t .. nginx: [emerg] invalid number of arguments in "echo_sleep" directive in .../agentzh-echo-nginx-module/t/servroot/conf/nginx.conf:40
t/echo-bogus.t .. 1/2
# Failed test 'TEST 1: echo unknown flag - exit status ok'
# at lib/Test/Nginx/Util.pm line 1507.
# got: '0'
# expected: '1'
# Looks like you failed 1 test of 2.
t/echo-bogus.t .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/2 subtests

Test Summary Report
-------------------
t/echo-bogus.t (Wstat: 256 Tests: 2 Failed: 1)
Failed test: 2
Non-zero exit status: 1
Files=1, Tests=2, 1 wallclock secs ( 0.02 usr 0.00 sys + 0.13 cusr 0.03 csys = 0.18 CPU)
Result: FAIL
```

With valgrind,

```
% TEST_NGINX_USE_VALGRIND=1 TEST_NGINX_BINARY=nginx/objs/nginx prove -l -r t/echo-bogus1.t
t/echo-bogus1.t .. skipped: must_exit is not compatible with extra checks
Files=1, Tests=0, 0 wallclock secs ( 0.02 usr 0.01 sys + 0.11 cusr 0.01 csys = 0.15 CPU)
Result: NOTESTS
```