BALL  1.5.0
Macros
Class Black Box Testing

Macros

#define PRECISION(a)    TEST::precision = (a);
 
#define START_TEST(class_name)
 
#define END_TEST
 
#define CHECK(test_name)
 
#define STATUS(message)
 
#define RESULT
 
#define SLEEP_FOR_MSECS(ms)   std::this_thread::sleep_for(std::chrono::milliseconds(ms))
 
#define NEW_TMP_FILE(filename)
 
#define NEW_TMP_FILE_WITH_SUFFIX(filename, suffix)
 
#define TEST_REAL_EQUAL(a, b)
 
#define TEST_EQUAL(a, b)
 
#define TEST_NOT_EQUAL(a, b)
 
#define TEST_EXCEPTION(exception_type, command)
 
#define ABORT_IF(condition)    if (condition) break;
 
#define TEST_FILE(filename, templatename)
 
#define TEST_FILE_REGEXP(filename, templatename)
 
#define CAPTURE_OUTPUT_LEVEL(level)
 
#define CAPTURE_OUTPUT_LEVEL_RANGE(minlevel, maxlevel)
 
#define COMPARE_OUTPUT(text)
 

Detailed Description

To provide a maximum reliability for all BALL classes, each class provides its own test program to ensure that each class compiles and behaves (at least basically) as intended.

The testprograms reside in the directory source/TEST, they may be built and executed by calling make test.

Each test program prints after execution either "PASSED" or "FAILED". If any of the subtest contained in the test program fails, the whole test failed. The result of the test program can also be checked via its exit code. An exit code of zero means "PASSED", non-zero exit code means "FAILED".

There are several macros defined to simplify the creation of a test program and to provide a common interface. Each test program consists of several subtests which usually test one method or property of the class. Each of this subtests uses a series of elementary tests to check the functionality of the method.

A number of elementary tests has been implemented that is sufficient for most cases:

To create a new test program, use the file source/TEST/Skeleton_test.C

Macro Definition Documentation

◆ ABORT_IF

#define ABORT_IF (   condition)     if (condition) break;

Skip remainder of subtest. If the condition is not fulfilled, the remainder of the test is skipped. The status (whether it fails or passes) remains unchanged.

Definition at line 507 of file classTest.h.

◆ CAPTURE_OUTPUT_LEVEL

#define CAPTURE_OUTPUT_LEVEL (   level)
Value:
{\
std::ostringstream TEST_strstr;\
Log.remove(std::cout);\
Log.remove(std::cerr);\
Log.insert(TEST_strstr, level, level);

Redirect output to the global logging facility. This macro (together with COMPARE_OUTPUT ) can be used to ensure that a function prints an error message to the global logging facility Log . It disables the output to cout and cerr and redirects all output to level to a temporary ostringstream. The contents of this stream can be compared with the expected output afterwards using the macro COMPARE_OUTPUT . Each CAPTURE_OUTPUT requires exactly one subsequent COMPARE_OUTPUT macro.

Definition at line 709 of file classTest.h.

◆ CAPTURE_OUTPUT_LEVEL_RANGE

#define CAPTURE_OUTPUT_LEVEL_RANGE (   minlevel,
  maxlevel 
)
Value:
{\
std::ostringstream TEST_strstr;\
Log.remove(std::cout);\
Log.remove(std::cerr);\
Log.insert(TEST_strstr, minlevel, maxlevel);

Redirect output to the global logging facility. This macro (together with COMPARE_OUTPUT ) can be used to ensure that a function prints an error message to the global logging facility Log . It disables the output to cout and cerr and redirects all output to level to a temporary ostringstream. The contents of this stream can be compared with the expected output afterwards using the macro COMPARE_OUTPUT . Each CAPTURE_OUTPUT requires exactly one subsequent COMPARE_OUTPUT macro.

Definition at line 728 of file classTest.h.

◆ CHECK

#define CHECK (   test_name)
Value:
TEST::test = true;\
TEST::newline = false;\
if (TEST::verbose > 0)\
std::cout << "checking " << #test_name << "... " << std::flush;\
try\
{\
while (true)\
{\
bool newline
Definition: classTest.h:25
bool test
Definition: classTest.h:20
int verbose
Definition: classTest.h:18

Declare subtest name. This macro is used to declare the name of a subtest. If you want to check e.g. the setName method of a class, insert a line

CHECK(setName)#
in your test program. If the test program is called in verbose mode, this leads to the name of the subtest being printed on execution.
This macro also opens a try block to catch any unexpected exceptions thrown in the course of a subtest. To catch wanted exceptions (i.e. to check for exceptions that are the expected result of some command) use the TEST_EXCEPTION macro. The try block opened by CHECK is closed in RESULT , so these two macros have to be balanced.

Definition at line 202 of file classTest.h.

◆ COMPARE_OUTPUT

#define COMPARE_OUTPUT (   text)
Value:
Log.remove(TEST_strstr);\
Log.insert(std::cout, LogStream::INFORMATION_LEVEL, LogStream::ERROR_LEVEL - 1);\
Log.insert(std::cerr, LogStream::ERROR_LEVEL);\
TEST::this_test = (::strncmp(TEST_strstr.str().c_str(), text, TEST_strstr.str().size()) == 0);\
\
if ((TEST::verbose > 1) || (!TEST::this_test && (TEST::verbose > 0)))\
{\
/* reserve space for the null-terminated content of the strstrem */\
char* TEST_strstr_contents = new char[TEST_strstr.str().size() + 1];\
::strncpy(TEST_strstr_contents, TEST_strstr.str().c_str(), TEST_strstr.str().size());\
TEST_strstr_contents[TEST_strstr.str().size()] = '\0';\
\
{\
std::cout << std::endl;\
}\
std::cout << " (line " << __LINE__ << " COMPARE_OUTPUT(" << #text << "): got '" << (TEST_strstr_contents) << "', expected '" << (text) << ") ";\
std::cout << (TEST::this_test ? " + " : " - ") << std::endl;\
delete [] TEST_strstr_contents;\
}\
}
BALL_EXPORT LogStream Log
bool this_test
Definition: classTest.h:21
void remove(std::ostream &s)

Compare output made to the global logging facility.

See also
CAPTURE_OUTPUT

Definition at line 739 of file classTest.h.

◆ END_TEST

#define END_TEST
Value:
/* global try block */\
}\
catch (std::exception& e) { TEST::printException(e, false); }\
catch (...) { TEST::printException(0, false); }\
\
/* clean up all temporary files */\
while (TEST::tmp_file_list.size() > 0 && TEST::verbose < 1)\
{\
}\
/* check for exit code */\
std::cout << (TEST::all_tests ? "PASSED" : "FAILED") << std::endl;\
return !TEST::all_tests;\
}\
void printException(T e, bool subtest)
Definition: classTest.h:87
list< string > tmp_file_list
Definition: classTest.h:26
bool all_tests
Definition: classTest.h:19
bool remove()

Termination of test program. This macro implements the correct termination of the test program and should therefore be the last macro to call. It determines the exit code based on all previously run subtests and prints out the message "PASSED" or "FAILED". This macro also closes the global try block opened by START_TEST and contains the related catch clauses. If an exception is caught here, the test program fails.

Definition at line 168 of file classTest.h.

◆ NEW_TMP_FILE

#define NEW_TMP_FILE (   filename)
Value:
TEST::tmp_file_list.push_back(filename);\
if (TEST::verbose > 1)\
{\
if (!TEST::newline) \
{\
std::cout << std::endl;\
}\
std::cout << " creating new temporary file '" << filename << "' (line " << __LINE__ << ")" << std::endl;\
}\
static bool createTemporaryFilename(String &temporary, const String &suffix=".TMP")

Create a temporary filename. This macro assigns a new temporary filename to the string variable given as its argument. The filename is created using File::createTemporaryFilename . All temporary files are deleted if END_TEST is called.

Parameters
filenameString will contain the filename on completion of the macro

Definition at line 285 of file classTest.h.

◆ NEW_TMP_FILE_WITH_SUFFIX

#define NEW_TMP_FILE_WITH_SUFFIX (   filename,
  suffix 
)
Value:
TEST::tmp_file_list.push_back(filename);\
if (TEST::verbose > 1)\
{\
if (!TEST::newline) \
{\
std::cout << std::endl;\
}\
std::cout << " creating new temporary file '" << filename << "' (line " << __LINE__ << ")" << std::endl;\
}\

Create a temporary filename with given suffix. This macro assigns a new temporary filename to the string variable given as its argument. The filename is created using File::createTemporaryFilename . All temporary files are deleted if END_TEST is called.

Parameters
filenameString will contain the filename on completion of the macro

Definition at line 305 of file classTest.h.

◆ PRECISION

#define PRECISION (   a)     TEST::precision = (a);

Define the precision for floating point comparisons. The macro TEST_REAL_EQUAL checks whether the floating point number returned by the subtest is close to the expected result by comparing the absolute value of the difference of the two values to PRECISION.

The default value is $10^{-6}$. It is possible to redefine precision in the test program by calling this macro with the new value.

Definition at line 112 of file classTest.h.

◆ RESULT

#define RESULT
Value:
break;\
}\
}\
/* catch FileNotFound exceptions to print out the file name */\
catch (std::exception& e) { TEST::printException(e, true); }\
catch (...) { TEST::printException(0, true); }\
\
if (TEST::verbose > 0){\
std::cout << " ";\
std::cout << (TEST::test ? "passed" : "FAILED") << std::endl;\
}\

Check subtest result. Each elementary test macro updates an internal variable (TEST, defined by START_TEST ) that holds the state of the current subtest.

RESULT prints whether the subtest has failed or passed in verbose mode and updates the internal variables TEST::all_tests that describes the state of the whole class test. TEST::all_tests is initialized to be true. If any elementary test fails, TEST::test becomes false. At the time of the next call to RESULT, TEST::all_tests will be set to false, if TEST::test is false. One failed elementary test leads therefore to a failed subtest, which leads to a failed class test.
This macro closes the try block opened by CHECK, so CHECK and RESULT have to be balanced, or some ugly compile-time errors may occur. RESULT first tries to catch all BALL exceptions (i.e. exceptions derived from GeneralException). If this fails, it tries to catch any exception. After the exception is thrown, the execution will continue with the next subtest, the current subtest will be marked as failed (as is the whole test program).

Definition at line 254 of file classTest.h.

◆ SLEEP_FOR_MSECS

#define SLEEP_FOR_MSECS (   ms)    std::this_thread::sleep_for(std::chrono::milliseconds(ms))

Sleep macro. Block execution of the current block for at least ms milliseconds.

Parameters
mswaiting time in milliseconds

Definition at line 276 of file classTest.h.

◆ START_TEST

#define START_TEST (   class_name)
Value:
\
int main(int argc, char **argv)\
{\
\
if (argc == 2) {\
if (!strcmp(argv[1], "-v"))\
if (!strcmp(argv[1], "-V"))\
};\
\
if ((argc > 2) || ((argc == 2) && (TEST::verbose == 0))) {\
std::cerr << "Checks " #class_name " class" << std::endl;\
\
std::cerr << "On successful operation it simply returns OK," << std::endl;\
std::cerr << "otherwise FAILURE is printed." << std::endl;\
std::cerr << "If called with an argument of -v, " << argv[0] << " prints detailed" << std::endl;\
std::cerr << "information about individual tests." << std::endl;\
std::cerr << "Option -V provides verbose information on" << std::endl;\
std::cerr << "every subtest." << std::endl;\
return 1;\
}\
\
if (TEST::verbose > 0)\
std::cout << "Version: " << TEST::version_string << std::endl;\
\
try {\
int main(int argc, char **argv)
Definition: main.C:46
const char * version_string
Definition: classTest.h:24

Create the test header for a certain class. This macro defines the start of the test program for a given classname. The classname is printed together with some information when calling the test program with any arguments (except for #-v# or #-V#).

This macro should be the first to call in a test program. It introduces a global try block to catch any unwanted exceptions. If any of these exceptions occurs, all tests failed. Exceptions defined by BALL (i.e. exception classes derived from GeneralException ) provide some additional information that is evaluated by the END_TEST macro. The END_TEST macro also closes the try block. This try block should never catch an exception! All exceptions that are thrown due to some malfunction in one of the member functions should be caught by the try block created by CHECK and RESULT .

Definition at line 128 of file classTest.h.

◆ STATUS

#define STATUS (   message)
Value:
if (TEST::verbose > 1)\
{\
if (!TEST::newline) \
{\
std::cout << std::endl;\
}\
std::cout << " status (line " << __LINE__ << "): " << message << std::endl;\
}\

Print a status message. If tests require longer preparations, STATUS may be used to print some intermediated progress messages. STATUS uses cout to print these messages (in verbose mode only). The given stream expression message is prefixed by the string status: and terminated with a newline. All valid operations on a stream may be performed in message.

Example:
STATUS("just calculated x = " << setprecision(10) << x)

Definition at line 225 of file classTest.h.

◆ TEST_EQUAL

#define TEST_EQUAL (   a,
 
)
Value:
{\
TEST::this_test = ((a) == (b));\
if ((TEST::verbose > 1) || (!TEST::this_test && (TEST::verbose > 0)))\
{\
{\
std::cout << std::endl;\
}\
std::cout << " (line " << __LINE__ << " TEST_EQUAL(" << #a << ", " << #b << "): got " << (a) << ", expected " << (b) << ") ";\
std::cout << (TEST::this_test ? " + " : " - ") << std::endl;\
}\
}\

Generic equality macro. This macro uses the operator == to check its two arguments for equality. Besides handling some internal stuff, it basically evaluates #((a) == (b))#.

Remember that operator == has to be defined somehow for the two argument types.
Parameters
avalue/object to test
bexpected value

Definition at line 349 of file classTest.h.

◆ TEST_EXCEPTION

#define TEST_EXCEPTION (   exception_type,
  command 
)

Exception test macro. This macro checks if a given type of exception occurred while executing the given command. Example:

TEST_EXCEPTION(Exception::IndexOverflow, vector3[-1])#
If no or a wrong exception occurred, false is returned, otherwise true.
Parameters
exception_typethe exception-class
commandany general C++ or BALL-specific command

Definition at line 399 of file classTest.h.

◆ TEST_FILE

#define TEST_FILE (   filename,
  templatename 
)

File comparison macro. This macro is used to test file operations. It compares the file with name filename against a template file templatename. Corresponding lines of the two files have to be identical.

See also
TEST_FILE_REGEXP for more sophisticated comparisons

Definition at line 517 of file classTest.h.

◆ TEST_FILE_REGEXP

#define TEST_FILE_REGEXP (   filename,
  templatename 
)

Regular expression file comparison macro. This macro is used to test file operations. It compares the file with name filename against a template file templatename. Each line of the template file starting with `‘/’' is considered to contain a regular expression, which has to match the corresponding line in the input file. All other lines of the input and the template file have to be identical.

Definition at line 601 of file classTest.h.

◆ TEST_NOT_EQUAL

#define TEST_NOT_EQUAL (   a,
 
)
Value:
{\
TEST::this_test = !((a) == (b));\
if ((TEST::verbose > 1) || (!TEST::this_test && (TEST::verbose > 0)))\
{\
{\
std::cout << std::endl;\
}\
std::cout << " (line " << __LINE__ << " TEST_NOT_EQUAL(" << #a << ", " << #b << "): got " << (a) << ", forbidden is " << (b) << ") ";\
std::cout << (TEST::this_test ? " + " : " - ") << std::endl;\
}\
}\

Generic inequality macro. This macro checks for inequality as TEST_EQUAL tests for equality. The only difference between the two macros is that TEST_NOT_EQUAL evaluates #!((a) == (b))#.

Parameters
avalue/object to test
bforbidden value

Definition at line 373 of file classTest.h.

◆ TEST_REAL_EQUAL

#define TEST_REAL_EQUAL (   a,
 
)
Value:
TEST::this_test = fabs((a) - (b)) < TEST::precision; \
if ((TEST::verbose > 1) || (!TEST::this_test && (TEST::verbose > 0)))\
{\
{\
std::cout << std::endl;\
}\
std::cout << " (line " << __LINE__ << " TEST_REAL_EQUAL("<< #a << ", " << #b << "): got " << (a) << ", expected " << (b) << ") ";\
std::cout << (TEST::this_test ? " + " : " - ") << std::endl;\
}\
double precision
Definition: classTest.h:30

Floating point equality macro. Checks whether the absolute value of the difference of the two floating point values a and b is less or equal to the value defined by PRECISION .

Parameters
afloating point value to test
bexpected value

Definition at line 325 of file classTest.h.