Package: nala
🦁 NalaA delightful test framework forCprojects.Based onNarwhalandNarmock.FeaturesAutomatic test discoveryUse the same generic assertions everywhereAssertion failures reported as diffsPowerful mockingEasy-to-use output capturing utilitiesTraceback(s) on failureWorks well with errors reported by sanitizersTest isolation withfork()Only run tests matching given patternAutomatic build and run on file changeOptional parallel test execution to save timeAmalgamated source and header files ready to drop in your projectJSON test reportInstallationIt’s recommended to installNalawithpip. Only Python 3 is
supported.$pipinstallnalaAlternatively, if mocking is not needed, you can download theamalgamatedheader and source files:nala.hnala.cDrop the two files in your project, make surenala.cis compiled
and linked just like the other source files of your test program and
you should be good to go.Getting startedUsenala init footo create a folderfoowith two test files,test_assertions.candtest_time.c. The first uses all
assertions and captures output, and the second mocks the time
function.$nalainitfooRun'make -C foo'tobuildandrunalltests!The assertions tests looks like this:#include"nala.h"TEST(assertions){inti;intarray[]={1,5,2};ASSERT_EQ(NULL,NULL);ASSERT_NE(1,2);ASSERT_LT(1.0,2.0);ASSERT_LE(1,1);ASSERT_GT(2L,1L);ASSERT_GE(1,1);ASSERT_TRUE(true);ASSERT_FALSE(false);ASSERT_SUBSTRING("12345","34");ASSERT_NOT_SUBSTRING("12345","4567");ASSERT_MEMORY_EQ("abcd","abcd",5);ASSERT_FILE_EQ("test_assertions.c","test_assertions.c");ASSERT_ARRAY_EQ(array,array,sizeof(array));ASSERT_FUNCTION_POINTER_EQ(assertions,assertions);ASSERT_FUNCTION_POINTER_NE(assertions,NULL);ASSERT(1==1);CAPTURE_OUTPUT(output,errput){printf("output!\n");fprintf(stderr,"errput!\n");}ASSERT_EQ(output,"output!\n");ASSERT_EQ(errput,"errput!\n");for(i=0;i<3;i++){WITH_MESSAGE("i: %d",i){ASSERT_EQ(array[i],array[i]);}}}And the time tests:#include<time.h>#include"nala.h"TEST(mock_time){time_mock_once(42);ASSERT_EQ(time(NULL),42);}Build and run all tests withmake-s-Cfoo.Build all tests but only run those whose name containstime. This
is done by givingARGS=time.Runmake-s-Cfoo autoto build and run all tests automatically
when any source file in your project is modified. With the command
running, make the time test fail by returning41instead of42from the time mock.#include<time.h>#include"nala.h"TEST(mock_time){time_mock_once(41);ASSERT_EQ(time(NULL),42);}Notice how the test is built and run automatically.Create a code coverage report and open it with Firefox.$ make -s -C foo coverage
Code coverage report: /home/erik/workspace/nala/foo/build/coverage/index.html
$ firefox /home/erik/workspace/nala/foo/build/coverage/index.htmlAPIBelow is a list of all macros and functions in the API. They are all
defined/declared ininclude/nala.h.AssertionsASSERT_EQ(actual,expected);// Assert that given characters, numbers, pointers or// strings are equal.ASSERT_NE(actual,expected);// Assert that given characters, numbers, pointers or// strings are not equal.ASSERT_LT(actual,expected);// Assert that actual is less than expected.ASSERT_LE(actual,expected);// Assert that actual is less than or equal to expected.ASSERT_GT(actual,expected);// Assert that actual is greater than expected.ASSERT_GE(actual,expected);// Assert that actual is greater than or equal to// expected.ASSERT_TRUE(actual);// Assert that given value is true.ASSERT_FALSE(actual);// Assert that given value is false.ASSERT_SUBSTRING(haystack,needle);// Assert that given haystack string contains given// needle string.ASSERT_NOT_SUBSTRING(haystack,needle);// Assert that given haystack string does not contain// given needle string.ASSERT_MEMORY_EQ(actual,expected,size);// Assert that given memory regions are equal.ASSERT_FILE_EQ(actual,expected);// Assert that given files are equal.ASSERT_ARRAY_EQ(actual,expected,size);// Assert that given arrays are equal.ASSERT_FUNCTION_POINTER_EQ(actual,expected);// Assert that given function pointers are equal.ASSERT_FUNCTION_POINTER_NE(actual,expected);// Assert that given function pointers are not equal.ASSERT(cond);// Assert that given condition is true.Other macrosFAIL(message);// Fail current test with given message.CAPTURE_OUTPUT(stdout_name,stderr_name);// A capture output block.WITH_MESSAGE(format,...);// Additional message on error block. May be nested.Functionsvoid*nala_alloc(size_tsize);// Allocate a memory buffer of given size that is// automatically freed after the test. Always returns// a valid pointer.voidnala_auto_free(void*buf_p);// Automatically free given buffer after the test.// free() is called to free the buffer.voidnala_exit(intstatus);// Performs post-test checks and cleanup, and then// exits with status 0. The status parameter is// ignored.MockingGenerating mocksThenala generate_mockscommand finds mocked functions in your
test code and generatesnala_mocks.h,nala_mocks.candnala_mocks.ldflags. The first two files declare and define mocks,
while the last file contains linker flags.Use--rename-parameters-fileto rename function parameters, often
useful when mocking standard library functions. If not given, Nala
renamesa few function parametersby default. Also, any__-prefix is removed from all parameters.Use--no-rename-parametersnot to rename any function
parameters. Overrides--rename-parameters-file.Use--no-implementationwhen the implementation of functions
matching given Unix shell-style wildcards pattern are not available in
the binary (and therefore should not be called by the generated code,
as the linker would give a relocation error). This option may be given
multipe times. An alternative to using this option is to manually
implement the missing functions. Here is an example implementation offoo()that makes the test fail if called.intfoo(){FAIL("No real implementation available!\n");return(0);}Use--no-real-variadic-functionsnot to add any real variadic
functions. Nala addsa few variadic functionsby default, given
that they are mocked.Here is an example of how to generate mocks:$nalacat*.c|gcc-DNALA_GENERATE_MOCKS-xc-E-|nalagenerate_mocksnala cat *.cshould only concatenate test source files, not any
other source files in your project.Nala requires test source code to be expanded by the preprocessor. You
can directly pipe the output ofgcc-DNALA_GENERATE_MOCKS-xc-E-to the command-line utility.Mocking object-internal function callsThe GNU linkerldwrap feature (--wrap=<symbol>) does not wrap
object-internal function calls. As Nala implements mocking by wrapping
functions, object-internal function calls can’t be mocked just using
the linker. To mock these, after compilation, runnala wrap_internal_symbols ...for each object file, and then pass them
to the linker.Also, local (static) functions can’t be mocked, only global
functions can!%.o:%.c$(CC)-o$@$<nalawrap_internal_symbolsnala_mocks.ldflags$@Mock APIA function mock will call the real implementation by default. Use the
functions below to control mock object behaviour.Variadic functions willnotcall the real implementation by
default. Give--implementationtonala generate_mocksto
generate calls to the real function (taking ava_listinstead of...).There are plenty of mock-examples in theexamples folder. All
inline examples below can also be found in themock_api_examplesexample.For all functions<params>is all char-pointer (string) and primitive type
parameters of the mocked function.Same behaviour for every call.voidFUNC_mock(<params>,<res>);// check parameters and returnvoidFUNC_mock_ignore_in(<res>);// ignore parameters and returnvoidFUNC_mock_none();// no calls allowedvoidFUNC_mock_implementation(*);// replace implementationvoidFUNC_mock_real();// real implementationAn example:/* int foo(int value); */TEST(foo_every_call){foo_mock(1,2);/* All calls to foo() expects its parameter to be 1 and returns 2. */ASSERT_EQ(foo(1),2);ASSERT_EQ(foo(1),2);}Per call control.intFUNC_mock_once(<params>,<res>);// check parameters and return once (per call)// returns a mock instance handleintFUNC_mock_ignore_in_once(<res>);// ignore parameters and return once (per call)// returns a mock instance handlevoidFUNC_mock_real_once();// real implementation once (per call)An example:/* int foo(int value); */TEST(foo_per_call){foo_mock_once(1,2);foo_mock_once(4,5);/* First call to foo() expects its parameter to be 1 and returns 2. */ASSERT_EQ(foo(1),2);/* Second call to foo() expects its parameter to be 4 and returns 5. */ASSERT_EQ(foo(4),5);/* Third call will fail and the test will end. */foo(10);}Changes the behaviour of currect mock object (most recent*_mock()or*_mock_once()call). Works for both per call and every call
functions above.voidFUNC_mock_set_errno(int);// errno on return, 0 by defaultvoidFUNC_mock_set_callback(*);// additional checks and/or actions// called just before returning from the mockAn example:/* int foo(int value); */TEST(foo_set_errno){foo_mock_once(1,2);foo_mock_set_errno(EINVAL);ASSERT_EQ(foo(1),2);ASSERT_EQ(errno,EINVAL);}Get per call input parameters.*FUNC_mock_get_params_in(int);// get input parameters for given mock instance// handleAn example:/* typedef void (*callback_t)(void); *//* void bar(callback_t callback); */staticvoidfie(void){printf("fie() called!\n");}TEST(bar_get_params_call_callback){inthandle;handle=bar_mock_once();bar(fie);/* Call the callback (calls fie()). */bar_mock_get_params_in(handle)->callback();}For pointer and array function parametersChanges the behaviour of currect mock (most recent*_mock()or*_mock_once()call). Works for both per call and every call
functions above.voidFUNC_mock_set_PARAM_in(*,size_t);// check on inputvoidFUNC_mock_set_PARAM_in_assert(*);// custom assert function on inputvoidFUNC_mock_set_PARAM_in_pointer(*);// check pointer (the address) on inputvoidFUNC_mock_set_PARAM_out(*,size_t);// value on returnvoidFUNC_mock_set_PARAM_out_copy(*);// custom output copy functionAn example:/* struct foo_t { char *string_p }; *//* void fum(int *value_p, struct foo_t *foo_p); */staticvoidassert_foo_string(structfoo_t*actual_p,structfoo_t*expected_p,size_tsize){ASSERT_EQ(size,sizeof(*expected_p));ASSERT_EQ(actual_p->string_p,expected_p->string_p);}TEST(fum_in_out){intvalue;structfoo_tfoo;fum_mock_once();/* Expect *value_p to be 1 when fum() is called, and assign 2 to
       it before returning. */value=1;fum_mock_set_value_p_in(&value,sizeof(value));value=2;fum_mock_set_value_p_out(&value,sizeof(value));/* Use a custom parameter assert function to check that
       foo_p->string_p is "Hello!"  when fum() is called. */foo.string_p="Hello!";fum_mock_set_foo_p_in(&foo,sizeof(foo));fum_mock_set_foo_p_in_assert(assert_foo_string);value=1;foo.string_p="Hello!";fum(&value,&foo);ASSERT_EQ(value,2);}For function parameters part of <params>Changes the behaviour of currect mock object (most recent*_mock()or*_mock_once()call). Works for both per call and every call
functions above.voidFUNC_mock_ignore_PARAM_in();// ignore on inputAn example:/* void foo(int value); */TEST(foo_ignore_value){foo_mock_once(1,2);foo_mock_ignore_value_in();ASSERT_EQ(foo(9),2);}For variadic functionsVariadic function mocks are slightly different from the above. Their
parameter list is extended with a format string (format) and an
ellipsis (...), as shown below.voidFUNC_mock(<params>,<res>,format,...);voidFUNC_mock_once(<params>,<res>,format,...);voidFUNC_mock_ignore_in(<res>,format);voidFUNC_mock_ignore_in_once(<res>,format);The format string supports the following specifiers.%d  - signed integer
%u  - unsigned integer
%ld - signed long integer
%lu - unsigned long integer
%p  - pointer address
%s  - stringThe%pspecifier takes no value when calling the mock function,
just like pointers are not part of the mock function parameters
list. Instead, set pointers after the mock call.foo_mock(3,"%d%p%s",5,"the-string");foo_mock_set_va_arg_in_pointer_at(1,NULL);The variadic parameters are controlled by index instead of name.voidFUNC_mock_ignore_va_arg_in_at(uint);// ignore on inputvoidFUNC_mock_set_va_arg_in_at(uint,*,size_t);// check on inputvoidFUNC_mock_set_va_arg_in_assert_at(uint,*);// custom assert function on inputvoidFUNC_mock_set_va_arg_in_pointer_at(uint,*);// check pointer (the address) on inputvoidFUNC_mock_set_va_arg_out_at(uint,*,size_t);// value on returnvoidFUNC_mock_set_va_arg_out_copy_at(uint,*);// custom output copy functionLimitationsStructs and unions passed by value are ignored.va_listparameters are ignored.malloc()andfree()can’t be mocked if forking and using
gcov. They probably can if wrapping__gcov_fork()in an
suspend/resume-block.staticfunctions can’t be mocked.Only primitive data type members are asserted when comparing
structs. Pointer and union members are not asserted, and probably
never will be.Debugging tipsNala executes each test in its own process. This means that following
the execution of a test with a debugger can be a bit tricky because
debuggers likeGDBcan only follow a single process at a time.If you’re usingGDB, set a breakpoint at<test>_before_forkand
then run the program until it stops at the breakpoint. Before
continuing the program execution, tellGDBto follow the forked test
process by settingfollow-fork-modetochild.Below are all commands to debug theassertionstest in the example
above.$ gdb foo/build/app
(gdb) b assertions_before_fork
(gdb) r
(gdb) set follow-fork-mode child
(gdb) cThegdbmake target startsGDBand runs all commmands listed
above. Set theTESTmake variable to the test to debug. BothTEST=test_assertions::assertionsandTEST=assertionsare
accepted. The test name may be partialy given, as long as it only
matches one test.$ make -s -C foo gdb TEST=assertionsThe test program takes optional arguments as below, which also can be
helpful when debugging, especially--print-all-calls.$ foo/build/app --help
usage: foo/build/app [-h] [-v] [-c] [-a] [-r] [-f] [-j] [<test-pattern>]

Run tests.

positional arguments:
  test-pattern                  Only run tests matching given pattern. '^' matches
                                the beginning and '$' matches the end of the test
                                name.

optional arguments:
  -h, --help                    Show this help message and exit.
  -v, --version                 Print version information.
  -c, --continue-on-failure     Continue on test failure.
  -a, --print-all-calls         Print all calls to ease debugging.
  -r, --report-json-file        JSON test report file (default: report.json).
  -f, --print-test-file-func    Print file:function for exactly one test.
  -j, --jobs                    Run given number of tests in parallel
                                (default: 1).Compiler flagsPass-no-pie-g-O0-fsanitize=addressto the compiler for better
error reporting.Read more about sanitizers here:https://en.wikipedia.org/wiki/AddressSanitizerOther unit test frameworksOther C unit test frameworks with similar feature set as Nala.CMock+UnitycmockaIdeasNew set of generated parameter functions where_in()functions are
renamed to_in_buffer(). New_in()functions are added,
without the size parameter. Also remove the size parameter from the
assert function, as it is seldom used.voidFUNC_mock_set_PARAM_in(*);// check on inputvoidFUNC_mock_set_PARAM_in_buffer(*,size_t);// check on inputvoidFUNC_mock_set_PARAM_in_assert(*);// custom assert function on inputvoidFUNC_mock_set_PARAM_in_pointer(*);// check pointer (the address) on inputvoidFUNC_mock_set_PARAM_out(*);// value on returnvoidFUNC_mock_set_PARAM_out_buffer(*,size_t);// value on returnvoidFUNC_mock_set_PARAM_out_copy(*);// custom output copy functionAn example:/* struct foo_t { char *string_p }; *//* void fum(int *value_p, struct foo_t *foo_p); */staticvoidassert_foo_string(structfoo_t*actual_p,structfoo_t*expected_p){/* Is size is needed (which is seldom is). */ASSERT_EQ(nala_mock_get_param_in_size(),sizeof(*expected_p));ASSERT_EQ(actual_p->string_p,expected_p->string_p);}TEST(fum_in_out){intvalue;structfoo_tfoo;fum_mock_once();/* Expect *value_p to be 1 when fum() is called, and assign 2 to
       it before returning. */value=1;fum_mock_set_value_p_in(&value);value=2;fum_mock_set_value_p_out(&value);/* Use a custom parameter assert function to check that
       foo_p->string_p is "Hello!"  when fum() is called. */foo.string_p="Hello!";fum_mock_set_foo_p_in(&foo);fum_mock_set_foo_p_in_assert(assert_foo_string);value=1;foo.string_p="Hello!";fum(&value,&foo);ASSERT_EQ(value,2);}