Dtest is a project mainly written in PYTHON and SHELL, based on the Apache-2.0 license.
Dependency-based Python testing framework
The DTest framework is a testing framework, similar to the standard
unittest
package provided by Python. The value-add for DTest,
however, is that test execution is threaded, through use of the
eventlet
package. The DTest package also provides the concept of
"dependencies" between tests and test fixtures--thus the "D" in
"DTest"--which ensure that tests don't run until the matching set up
test fixtures have completed, and that the tear down test fixtures
don't run until all the associated tests have completed. Dependencies
may also be used to ensure that tests requiring the availability of
certain functionality don't run if the tests of that specific
functionality fail.
The simplest test programs are simple functions with names beginning
with "test," located in Python source files whose names also begin
with "test." It is not even necessary to import any portion of the
DTest framework. If tests are collected in classes, however, or if
use of the more advanced features of DTest is desired, a simple from dtest import *
is necessary. This makes available the DTestCase
class--which should be extended by all classes containing tests--as
well as such decorators as @skip
and @nottest
.
Tests may be performed using the standard Python assert
statement;
however, a number of utility routines are available in the
dtest.util
module (also safe for import *
). Many of these
utility routines have names similar to methods of
unittest.TestCase
--e.g., dtest.util.assert_dict_equal()
is
analogous to unittest.TestCase.assertDictEqual()
.
The DTest framework supports test fixtures--set up and tear down
functions--at the class, module, and package level. Package-level
fixtures consist of functions named setUp()
and tearDown()
contained within "init.py" files; similarly, module-level fixtures
consist of functions samed setUp()
and tearDown()
within
modules containing test functions and classes of test methods. At the
class level, classes may contain setUpClass()
and
tearDownClass()
class methods (or static methods), which may
perform set up and tear down for each class. In all cases, the
setUp()
functions and the setUpClass()
method are executed
before any of the tests within the same scope; similarly, after all
the tests at a given scope have executed, the corresponding
tearDownClass()
method and tearDown()
functions are executed.
The DTest framework also supports per-test setUp()
and
tearDown()
functions or methods, which are run before and after
each associated test. For classes containing tests, each test
automatically has the setUp() and tearDown() methods of the class
associated with them; however, for all tests, these fixtures can be
explicitly set (or overridden from the class default). Consider the
following example::
@istest
def test_something():
# Test something here
pass
@test_something.setUp
def something_setup():
# Get everything set up ready to go...
pass
@test_something.tearDown
def something_teardown():
# Clean up after ourselves
pass
In this example, a DTest decorator (other than @nottest
) is
necessary preceding test_something()
; here we used @istest
,
but any other available DTest decorator could be used here. This
makes the @test_something.setUp
and @test_something.tearDown
decorators available. (For something analogous in the standard
Python, check out the built-in @property
decorator.)
Many test suites use test fixtures to set up temporary resources needed for a particular test. For instance, it's not uncommon for a fixture to set up a utility object, such as a server client, which could be reused by other tests. The DTest framework provides an alternative means of setting up such objects: test resources.
A test resource is any single object that may be required by a given
test. To create one, set up a class extending the Resource
class
and implement the setUp()
method on the class (and, optionally,
the tearDown()
method). There are two additional class attributes
that can be set. The first is oneshot
: if set to True, the
resource returned by setUp()
will only be used once, then
discarded. The second is dirtymeths
, which should contain a list
of methods which, when called, will cause the object to become
"dirty", causing it to be discarded after the test. (Setting or
deleting object attributes will also cause the object to be marked as
"dirty".)
To mark that a test requires a particular resource, use the
@require()
decorator; this decorator takes keyword arguments,
where the keys will be taken as the names of arguments to the test,
and the values must be instances of subclasses of Resource
. When
the test is run, DTest will create the actual resource objects and
pass them to the test as keyword arguments. As long as the object
does not become dirty, it will be reused for subsequent tests, subject
to threading constraints (resource objects may only be used by one
thread at a time).
Resources are subject to one limitation: the object actually passed to
the test is a proxy object which delegates attribute accesses to the
actual object allocated by the setUp()
method. Because of
optimizations within Python itself, it is not possible for this proxy
object to properly delegate special methods, such as __getitem__()
or __add__()
. Because of this, it is possible to retrieve the
true resource object, using the getobject()
function. Because
this removes the ability of the resources system to determine if the
resource becomes dirty, the dirty()
and clean()
functions are
also provided. Finally, to prevent an access from marking the object
as dirty, the cleanaccess()
function can be used in conjunction
with the with
statement like so::
with cleanaccess(resource):
resource.attribute = "some value"
Without the with
statement, this attribute setting would cause the
resource to be marked as dirty, but the with
inhibits this. Note
that it is legal to nest calls to cleanaccess()
, if necessary.
Resources may be specified with options, which should be
string-coercible constants. Any positional or keyword arguments
passed to the Resource
constructor will be saved and passed to the
setUp()
method when a resource must be constructed. In addition,
the resource caching mechanism uses these options to ensure that a
test is only passed resources with matching options.
tearDown()
If some special cleanup is needed for a resource, implement the
tearDown()
method on your Resource
subclass. It should take
two arguments: the object that was returned by setUp()
, and the
status of the test. For most resources, unless a test renders them
dirty, the status will be None
, and tearDown()
will be called
after all tests have run to completion; however, for resources which
have oneshot
set to True, the status should never be None
.
One possible use case for this is a test which uses temporary files,
which should be cleaned up after the test passes; should the test
fail, it may be useful to leave the temporary file around for
debugging purposes.
Running tests using the DTest framework is fairly straight-forward. A
script called run-dtests
is available. By default, the current
directory is scanned for all modules or packages whose names begin
with "test"; the search also recurses down through all packages. (A
"package" is defined as a directory containing "init.py".) Once
all tests are discovered, they are then executed, and the results of
the tests emitted to standard output.
Several command-line options are available for controlling the
behavior of run-dtests
. For instance, the "--no-skip" option will
cause run-dtests
to run all tests, even those decorated with the
@skip
decorator, and the "-d" option causes run-dtests
to
search a specific directory, rather than the current directory. For a
full list of options, use the "-h" or "--help" option.
Running run-dtests
from the command line is not the only way to
run tests, however. The run-dtests
script is a very simple script
that parses command-line options (using the OptionParser
constructed by the dtest.optparser()
function), converts those
options into a set of keyword arguments (using
dtest.opts_to_args()
), then passes those keyword arguments to the
dtest.main()
function. Users can use these functions to build the
same functionality with user-specific extensions, such as providing an
alternate DTestOutput instance to control how test results are
displayed, or providing an alternate method for controlling which
tests are skipped. See the documentation strings for these functions
and classes for more information.