Documenting cfme_tests


In addition to PEP 257, inline documentation of the cfme_tests code adheres to the Google Python Style Guide. The Google-recommended docstring format is very easy to both read and write, and thanks to the cartouche library, it’s parseable by sphinx, which we use to generate our documentation.

The documentation is built and hosted by the excellent readthedocs service, but should be built locally before making a pull request.


The napoleon library parses our docstrings and turns them into nicely rendered API docs in the sphinx output. As such, we should follow napoleon’s usage guidelines when writing docstrings:

According to PEP 257, docstrings should use triple double-quotes, not triple single-quotes (“”” vs. ‘’‘).


"""This is a docstring."""

'''This is not a docstring.'''

Documenting Tests

Tests are documented slightly differently to modules, in that they require certain extra information that isn’t required for a module/class/function. If a test uses the testgen library it must also specify a test_flag in the metadata section. An example of this is shown below.

"""Tests provisioning via PXE

    test_flag: pxe, provision

These flags are also defined in the cfme_data.yaml file, under the test_flags: key. A provider in the cfme_data.yaml can opt out of collection for a particular test_flag by including the flag in the list of excluded_test_flags: key in the providers stanza. All of the flags listings are listed in comma separated format. This was chosen to cut down on syntax characters and all values are whitespace stripped.

For a test to be collected for a provider:
  • the test_flag must be listed in the cfme_data.yaml file test_flags: key
  • the test_flag must be listed in the metadata section of the test’s docstring
  • the test_flag must NOT appear in the list of excluded_test_flags: for a particular provider

Linking new modules

As new modules are created, they’ll need to be added to the documentation tree. This starts in the toctree directive in docs/index.rst. Each entry in that tree references other .rst files in the docs/ directory, which can in turn reference documentation sources in their own toctree directives (ad infinitum).

Once the rst file has been inserted into the toctree (assuming one had to be created), sphinx needs to be told to generate documentation from the new code. We use sphinx’s autodoc feature to do this, and it looks like this:

.. automodule:: packagename.modulename

The paramater passed to the automodule should be the importable name of the module to be documented, cfme.login for example.

There is no hard and fast rule for where things should go in the toctree, but do try to keep the docs well-organized.

Building the Docs

Prior to pushing up new code, preview any new documentation by building to docs locally. You can do this using the sphinx-build command. From the cfme_tests directory:

sphinx-build -b html docs/ docs/build/

This will build html documentation based on the sources in the docs/ directory, and put them in the docs/build/ directory, which can then be opened in a browser:

google-chrome docs/build/index.html
# or...
firefox docs/build/index.html

Old and busted

The “legacy” code (contained mainly in the pages/ and tests/ directories) will not be documented here. Time spent documenting that code is better spent converting it to the new page style, in the cfme/ directory.