UI Coverage for a CFME/MIQ Appliance
simplecov can merge test results, but doesn’t appear to like working in a multi-process environment. Specifically, it clobbers its own results when running simultaneously in multiple processes. To solve this, each process records its output to its own directory (configured in coverage_hook). All of the individual process’ results are then manually merged (coverage_merger) into one big json result, and handed back to simplecov which generates the compiled html (for humans) and rcov (for jenkins) reports.
thing_toucher makes a best-effort pass at requiring all of the ruby files in the rails root, as well as any external MIQ libs/utils outside of the rails root (../lib and ../lib/util). This makes sure files that are never required still show up in the coverage report.
Pre-testing (pytest_configure hook):
- Add Gemfile.dev.rb to the rails root, then run bundler to install simplecov and its dependencies.
- Install and require the coverage hook (copy coverage_hook to config/, add require line to the end of config/boot.rb)
- Restart EVM (Rudely) to start running coverage on the appliance processes: killall -9 ruby; service evmserverd start
- TOUCH ALL THE THINGS (run thing_toucher.rb with the rails runner). Fork this process off and come back to it later
Post-testing (pytest_unconfigure hook):
- Poll thing_toucher to make sure it completed; block if needed.
- Stop EVM, but nicely this time so the coverage atexit hooks run: service evmserverd stop
- Run coverage_merger.rb with the rails runner, which compiles all the individual process reports and runs coverage again, additionally creating an rcov report
- Pull the coverage dir back for parsing and archiving
- For fun: Read the results from coverage/.last_run.json and print it to the test terminal/log
Post-testing (e.g. ci environment): 1. Use the generated rcov report with the ruby stats plugin to get a coverage graph 2. Zip up and archive the entire coverage dir for review
- class fixtures.ui_coverage.CoverageManager(ipappliance)¶
- class fixtures.ui_coverage.UiCoveragePlugin¶
- fixtures.ui_coverage.appliance_coverage_root = local('/var/www/miq/vmdb/coverage')¶
coverage root, should match what’s in the coverage hook and merger scripts
- fixtures.ui_coverage.rails_root = local('/var/www/miq/vmdb')¶
Corresponds to Rails.root in the rails env