agentzh/test-nginx — pull request #11 by maage (2014-04-13 18:31:27+00:00)
Title: feature: new block "---must_die"

Feature to test if exit value is something else than 0. Used to test
config parser can notice bogus config. If this is set, no requests
are done.

This is for #10.

Some changes to previous attempt. Mainly from review from @agentzh
- changed name from must_exit to must_die
- error_log works
- no need to set exit code

Valgrind / stap / profiling etc. are not supported. I think there is no point of running under those when you are expected to exit program readily. Maybe there should be additional checks for those. For now you can guard at test file instead, example below. And so every test should have must_exit or none should.

Quick test for echo module to demonstrate.

```
% cat t/echo-bogus.t
# vi:filetype=

use lib 'lib';
use Test::Nginx::Socket;

if ($ENV{TEST_NGINX_USE_VALGRIND} || $ENV{TEST_NGINX_CHECK_LEAK} || $ENV{TEST_NGINX_USE_STAP}) {
plan skip_all => 'must_exit is not compatible with extra checks';
}

repeat_each(1);

plan tests => repeat_each() * blocks() + 3;

run_tests();

__DATA__

=== TEST 1: echo unknown flag
--- config
location /echo {
echo -x hello;
}
--- must_exit
--- request
GET /echo

=== TEST 2: echo_sleep bad value
--- config
location /echo {
#echo_sleep -1;
echo_sleep;
echo hello;
}
--- must_die
--- request
GET /echo

# This test should fail
=== TEST 3: echo_sleep bad value
--- config
location /echo {
#echo_sleep -1;
echo_sleep;
echo hello;
}
--- must_die: 2
--- request
GET /echo

=== TEST 4: echo_sleep bad value
--- config
location /echo {
#echo_sleep -1;
echo_sleep;
echo hello;
}
--- must_die: 1
--- request
GET /echo

=== TEST 5: echo_sleep bad value
--- config
location /echo {
#echo_sleep -1;
echo_sleep;
echo hello;
}
--- must_die
--- error_log: emerg
--- request
GET /echo

# This test should fail
=== TEST 6: echo_sleep bad value
--- config
location /echo {
#echo_sleep -1;
echo_sleep;
echo hello;
}
--- must_die
--- error_log: humppa
--- request
GET /echo

=== TEST 7: echo_sleep bad value
--- config
location /echo {
#echo_sleep -1;
echo_sleep;
echo hello;
}
--- must_die
--- error_log: invalid number of arguments in "echo_sleep" directive
--- request
GET /echo
```

Test run:
```
% TEST_NGINX_BINARY=nginx/objs/nginx prove -r t/echo-bogus.t
t/echo-bogus.t .. nginx: [emerg] invalid number of arguments in "echo_sleep" directive in .../agentzh-echo-nginx-module/t/servroot/conf/nginx.conf:40
t/echo-bogus.t .. 1/10
# Failed test 'TEST 1: echo unknown flag - died as excpected'
# at lib/Test/Nginx/Util.pm line 1521.
# got: '0'
# expected: anything else
nginx: [emerg] invalid number of arguments in "echo_sleep" directive in.../agentzh-echo-nginx-module/t/servroot/conf/nginx.conf:40

# Failed test 'TEST 3: echo_sleep bad value - died as expected'
# at lib/Test/Nginx/Util.pm line 1507.
# got: '1'
# expected: '2'
nginx: [emerg] invalid number of arguments in "echo_sleep" directive in .../agentzh-echo-nginx-module/t/servroot/conf/nginx.conf:40
t/echo-bogus.t .. 5/10 nginx: [emerg] invalid number of arguments in "echo_sleep" directive in .../agentzh-echo-nginx-module/t/servroot/conf/nginx.conf:40
nginx: [emerg] invalid number of arguments in "echo_sleep" directive in .../agentzh-echo-nginx-module/t/servroot/conf/nginx.conf:40

# Failed test 'TEST 6: echo_sleep bad value - pattern "humppa" matches a line in error.log (req 0)'
# at lib/Test/Nginx/Socket.pm line 1007.
nginx: [emerg] invalid number of arguments in "echo_sleep" directive in .../agentzh-echo-nginx-module/t/servroot/conf/nginx.conf:40
# Looks like you failed 3 tests of 10.
t/echo-bogus.t .. Dubious, test returned 3 (wstat 768, 0x300)
Failed 3/10 subtests

Test Summary Report
-------------------
t/echo-bogus.t (Wstat: 768 Tests: 10 Failed: 3)
Failed tests: 3-4, 8
Non-zero exit status: 3
Files=1, Tests=10, 1 wallclock secs ( 0.03 usr 0.00 sys + 0.15 cusr 0.09 csys = 0.27 CPU)
Result: FAIL
[3] 12663 exit 1 TEST_NGINX_BINARY=nginx/objs/nginx prove -r t/echo-bogus.t
```

Test run with valgrind.

```
% TEST_NGINX_USE_VALGRIND=1 TEST_NGINX_BINARY=nginx/objs/nginx prove -r t/echo-bogus.t
t/echo-bogus.t .. skipped: must_exit is not compatible with extra checks
Files=1, Tests=0, 0 wallclock secs ( 0.02 usr 0.01 sys + 0.10 cusr 0.01 csys = 0.14 CPU)
Result: NOTESTS
```