Unit tests - a tutorial

My take on unit tests

There are many tutorials about unit testing on the Internet. This is one of them. To be more precise: this is a document about my simplistic understanding of unit tests[1] and about how I use them in my own small project. In this tutorial I use fragments of my code to demonstrate how unit tests can help a developer.

Please note that solutions presented here are very, very simple. Developers have been using unit tests for quite a long time now, and some of them decided to "scratch their own itch" and created good, comprehensive unit testing frameworks[2] that simplify employing unit tests in large, serious projects. If you intend to create new office package, or new web framework, or new operating system, then perhaps you should take a look at some more sophisticated solutions.

However if you are writing a small tool and don't want to add too much dependencies to your project, solutions from this tutorial may be of interest to you.

Author: Kamil Ignacak, acerion at wp dot pl
Version: 1
Date of last revision: 11.07.2011

Unit tests - an example

Suppose that you write a function:

int cdw_string_security_parser(char *input, char *output);

that should catch some "insecure" chars. Its simplified description is following:

if "input" string contains insecure character, put the character in "output" and return CDW_NO;
if "input" string contains no insecure characters, return CDW_OK;

Implementation of the function is easy, but you want to be sure that it works as you intended. Moreover, you want to be sure that if, for some reason, you modify implementation of the function, this modification won't break the function, i.e. that the function will still work correctly. How to achieve this? You write unit tests.

Here is the first unit test that you can write for a function specified above:

    char output[2];
    output[1] = '\0';
    int rv = cdw_string_security_parser("insecure character |", output);
    assert (rv == CW_NO);
    assert (!strcmp(output, "|"));

"|" is a character that you think is insecure in context of your application. The test works well, ensuring that:

the function detects invalid character "|" in input string and puts it in output buffer
the function will still work after upgrading its implementation; if the upgrade of implementation breaks the function, the function won't pass the test and you know that the upgrade introduced a bug;

The example shown above is very simple, and incomplete. It is very simple, because it assumes that there is only one insecure character. It is incomplete because it doesn't check special cases, e.g. a situation when input string consists only of insecure characters, or if the insecure character is at the beginning of input string. It is also incomplete because a string that doesn't contain any insecure chars isn't tested.

So you update your test units code:

    char output[2];
    output[1] = '\0';
    int rv;

    rv = = cdw_string_security_parser("insecure character |", output);
    assert (rv == CDW_NO);
    assert (!strcmp(output, "|"));

    rv = = cdw_string_security_parser("| insecure character", output);
    assert (rv == CDW_NO);
    assert (!strcmp(output, "|"));

    rv = = cdw_string_security_parser("|", output);
    assert (rv == CDW_NO);
    assert (!strcmp(output, "|"));

    rv = = cdw_string_security_parser("secure string", output);
    assert (rv == CDW_OK);

Unit test coverage has improved, you can be more confident that your function works as intended. Remember that unit testing won't discover all bugs in your software. In any reasonably large software there is at least one bug, and no amount of testing can give you 100% guarantee that you will find all bugs.

Top of page

Compacting unit test code

Example provided in previous point works just fine, but it has one downside: amount of code grows considerably as you add new input strings to test your function. Luckily in some cases this can be managed once you realise that your unit test consist of three elements:

input data
function call
checking result and comparing it with expected value

If your unit tests clearly follow this pattern you can rearrange your code using tables and loop. Like this:

   	/* input data + expected results, all put into one table;
	   one row in the table = one test case */
	struct {
		const char *input;   /* input data */
		const char *output;  /* expected result - part 1 */
		int rv;               /* expected result - part 2 */
	} test_data[] = {
		/* strings with insecure chars */
		{ "test*test",   "*",   CDW_NO },
		{ "*testtest",   "*",   CDW_NO },
		{ "testtest*",   "*",   CDW_NO },
		{ "test&test",   "&",   CDW_NO },
		{ "&testtest",   "&",   CDW_NO },
		{ "testtest&",   "&",   CDW_NO },
		{ "test\"test",  "\"",  CDW_NO },
		{ "\"testtest",  "\"",  CDW_NO },
		{ "testtest\"",  "\"",  CDW_NO },
		{ "test|test",   "|",   CDW_NO },
		{ "testtest|",   "|",   CDW_NO },
		{ "|testtest",   "|",   CDW_NO },
		{ "test$test",   "$",   CDW_NO },
		{ "testtest$",   "$",   CDW_NO },
		{ "$testtest",   "$",   CDW_NO },

		/* string without insecure chars */
		{ "test tes t-.,_/ =:598,_/fduf98-. _/  no:", (char *) NULL, CDW_OK },

		/* ending guard */
		{ (char *) NULL, (char *) NULL, CDW_CANCEL }};

	int i = 0;
	char output[2];
	output[1] = '\0';
	while (test_data[i].rv != CDW_CANCEL) {

		/* function call */
		int rv = cdw_string_security_parser(test_data[i].input, output);

		/* checking result and comparing it with expected value */
		assert (rv == test_data[i].crv);
		if (test_data[i].output != (char *) NULL) {
			assert (!strcmp(test_data[i].output, output));
		}

		i++;
	}

Now it is a lot easier to add new test cases. All it takes is to add one entry in input table.

The text above shows how you can create unit tests that check your code. Where to place such unit tests? How to structure them? How to invoke unit tests code?

Top of page

Where to put unit tests code?

Initially I thought that placing unit tests in a separate source code file would be a great idea: a module in one *.c file, and unit tests code, testing the module, in other *.c file. It worked just fine until I had to write unit tests for functions that were "private" functions of the module. Calling these private functions from other source code file would require me to expose the functions to the rest of a project: I would have to put their prototypes in *.h file, and make the functions globally visible, revealing details of implementation of a module. Separating unit tests from tested code turned out to be a bad idea.

Put code that tests functions from file 'A' in the same source file in which the functions are defined.

Top of page

How to organize unit test code in a file?

A module that you want to cover by unit tests usually contains few or more functions that you want to test. You may want to consider following scheme:

	/* two private functions that you want to test */
	int function_X(char *arg)
	{
	    ...
	}
	int function_Y(int arg)
	{
	    ...
	}


	/* public wrapper function for unit tests */
	int test_module_A_wrapper(void)
	{
	    int rv;
	    rv = test_function_X();
	    assert (rv == CDW_OK);
	    rv = test_function_Y();
	    assert (rv == CDW_OK);

	    return  CDW_OK;
	}

	/* two private functions implementing unit tests */
	int test_function_X(void)
	{
	    /* here you put unit tests for function_X */
	    ...
	    return  CDW_OK;
	}
	int test_function_Y(void)
	{
	    /* here you put unit tests for function_Y */
	    ...
	    return  CDW_OK;
	}

Now you have all this code in one file:

tested functions: function_X() and function_Y();
functions implementing unit tests: test_function_X() and test_function_Y();
wrapper function test_module_A_wrapper() - the only function that needs to be exposed by module 'A' in order to perform unit testing of the module;

Question: how to call the wrapper?

Top of page

How to organize unit tests in a project?

How to call all unit tests that you have coded in your project with just one command? The answer is pretty easy if you use autotools as your build system. The answer is "make check" - a build target, a command that you invoke in command line, similarly to simple "make" or "make clean".

Here is now a Makefile.am may look like if you want to add "make check" functionality to your build scripts:

           3          4                 13         14                 23          25         26            30

1 # all source files 2 prog_source_files = main.c module_A.c 5 # below we define two distinct build targets 6 ## target 1: 'prog' - program that will be installed in user's system 7 bin_PROGRAMS = prog 8 prog_SOURCES = $(prog_source_files) 9 # target 2: prog_tests - program that will be run in build directory 10 check_PROGRAMS = prog_tests 11 prog_tests_CPPFLAGS = -DENABLE_UNIT_TEST_CODE 12 prog_tests_SOURCES = $(prog_source_files) 15 # call prog_tests and catch its result 16 # source: 17 # http://www.freesoftwaremagazine.com/books/agaal/ 18 # automatically_writing_makefiles_with_autotools 19 check_SCRIPTS = greptest.sh 20 greptest.sh: 21 echo './prog_tests | grep "test result: success"' > greptest.sh 22 chmod +x greptest.sh 24 TESTS = $(check_SCRIPTS) 27 # CLEANFILES extends list of files that need to be removed when 28 # calling "make clean" 29 CLEANFILES = greptest.sh

Here are some key points in the file:

line 10: here we define an additional build target: after adding this line we will be able to enter "make check" in command line, and our build tool will build "prog_tests" program.
lines 19-22: whenever a "make check" is run, these lines create a greptest.sh script that runs our "prog_tests" and checks its returned value ("prog_tests" must print "test result: success" on success);
line 24 lists targets to be built and executed when a user enters "make check"

One interesting line is line 11: whenever a program with unit tests is being build, a ENABLE_UNIT_TEST_CODE flag is being defined. The flag can be used in source code files to decide which parts of source should be built/compiled. This feature is quite useful: you want to enable unit tests only in test builds, but not in production builds. There may be also other uses of the flag.

Top of page

References and resources

[1] Wikipedia article on unit testing.
[2] Wikipedia article on unit testing frameworks.
Chapter 4 of very nice article on Autotools. Scroll down to "Unit tests - supporting “make check”" section to see where some code from my Makefile.am comes from.
Syntax in code snippets has been highlighted using highlight tool.
Sample project using techniques described in this tutorial.

Top of page

Change Log

11.07.2011: initial version of the document: version 1

Copyright

Copyright (C) 2011 - 2016 Kamil Ignacak

This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

Top of page
Go back to cdw documentation

Unit tests - a tutorial

My take on unit tests

Table of contents: