Stability: [beta] Most of the functionality is there but it is still missing some testing and/or verification.
The Ciao assertion language (see The Ciao assertion language) allows writing tests (including unit tests) by means of test assertions. These assertions make it possible to write specific test cases at the predicate level. This library contains predicates that can be used to run tests in modules and gather or pretty-print the results. It also provides some special properties that are convenient when writing tests and the corresponding run-time support.
As described in The Ciao assertion language a test assertion is written as follows:
:- test predicate(A1, A2, ..., An) : <Precondition> => <Postcondition> + <Global properties> # <Comment>.
Where the fields of the test assertion have the usual meaning in Ciao assertions, i.e., they contain conjunctions of properties which must hold at certain points in the execution. Here we give a somewhat more operational (``test oriented'') reading to these fields:
The following are some example tests for a complex number evaluator (see Examples (unittest) for the full code):
:- module(ceval2, [ceval/2], [assertions, regtypes, nativeprops]). :- test ceval(A, B) : (A = c(3, 4) + c(1, 2) - c(2, 3)) => (B = c(2, 3)) + (not_fails, is_det). :- test ceval(A, B) : (A = c(3, 4) * c(1, 2) / c(1, 2)) => (B = c(3.0, 4.0)) + (not_fails, is_det). ceval(A, A) :- complex(A), !. ceval(A+B, C) :- ceval(A, CA), ceval(B, CB), add(CA, CB, C). ceval(A-B, C) :- ceval(A, CA), ceval(B, CB), sub(CA, CB, C). ceval(A*B, C) :- ceval(A, CA), ceval(B, CB), mul(CA, CB, C). ceval(A/B, C) :- ceval(A, CA), ceval(B, CB), div(CA, CB, C). ... :- regtype complex/1. :- export(complex/1). complex(c(A, B)) :- num(A), num(B).
Test assertions can be combined with other assertions:
:- test ceval(A, B) : (A = c(3, 4) + c(1, 2) - c(2, 3)) => (B = c(2, 3)) + (not_fails, is_det). :- test ceval(A, B) : (A = c(3, 4) * c(1, 2) / c(1, 2)) => (B = c(3.0, 4.0)) + (not_fails, is_det). :- check pred ceval/2 : gnd * term => gnd * complex.
Test assertions can also take the standard assertion status prefixes. In particular, a status of false can be used to state that a test fails. This can be useful to flag bugs as known.
:- false test ceval(A, B) : (A = c(3, 4) + c(1, 2) - c(2, 3)) => (B = c(2, 3)) + (not_fails, is_det).
Tests with a false (or true) prefix are not run.
There are some specific properties that only apply to testing which are provided in module unittest_props.pl (see Special properties for testing). For example, the limit to the number of solutions to be generated for the tested predicate can be set with the property try_sols(N), a timeout to a test can be set with the property timeout(N), times(N) specifies that the given test should be executed N times, etc.
The special property example can be used to mark the unit test as an example, so that it is documented as such in manuals. The default behavior in lpdoc is to not include the unit tests in manuals unless they are marked this way. For example, the following test would be included in the manual as an example:
:- test ceval(A, B) : (A = c(3, 4) + c(1, 2) - c(2, 3)) => (B = c(2, 3)) + (not_fails, is_det, example).
To run all these tests in a given bundle (as well as the other standard tests in the system) run the following (at the top level of the source tree or a bundle ):
ciao test
A convenient way to run these tests is by selecting options in the CiaoDbg menu within the development environment. This menu offers the following options:
The tests can also be run from the top level, loading this module (unittest.pl) and calling the appropiate predicates that it exports (see the module Usage and interface section below). This can also be done from a program, provided it imports this module.
These tests can be combined with the run-time checking of other assertions present in the involved modules. This can be done by including the rtchecks package in the desired modules. Any check assertions present in the code will then be checked dynamically during the execution of the tests and can detect additional errors.
If you need to write tests for predicates that are spread over several modules, but work together, it may be useful to create a separate module, and reexport the predicates required to build the tests. This allows performing integration testing, using the syntax and functionality of the test assertions.
Usage:show_untested_exp_preds(Alias)
Show any exported predicates that do not have test assertions. This is an aid towards ensuring that all exported predicates have tests.
Usage:run_tests_in_dir_rec(BaseDir,Opts,S)
Executes all the tests in the modules of the given directory and its subdirectories. You can indicate that the modules in a sub-directory should not be tested by placing an empty NOTEST file in that sub-directory. Also, if a NOTESTFILES file is present, containing patterns for modules, those modules will not be tested. Unittest's statistical summary is summarised in S, which is given value 0 if there are as many successes as expected, and 1 otherwise.
Usage:test_option(Opt)
Opt is a testing option.
Usage:test_action(Action)
Action is a testing action
Usage:show_test_output(Alias,Format)
Given a file Alias, tries to look up the respective unittest output file and print it to the standard output in the specified Format ('output' for test full trace, 'stats' for a statistical summary only, 'full' for both), otherwise emits a warning message that no test output file is avaiable.
Usage:run_tests_in_module(Alias,Opts,TestSummaries)
Run the tests in module Alias (with options Opts). The results of the tests are returned as data in TestSummaries. TestSummaries can be pretty printed using show_test_summaries/1 and statistical_summary/2.
Usage:run_tests_in_module(Alias,Opts)
Run the tests in module Alias. The results of the tests are printed out.
Usage:run_tests_in_module(Alias)
Run the tests in module Alias (using default options). The results of the tests are printed out.
Usage:run_tests_related_modules(Alias)
Usage:define_flag(Flag,FlagValues,Default)