rapid7/metasploit-framework — pull request #4332 by limhoff-r7 (2014-12-08 15:03:51+00:00)
Title: Test optimization

MSP-11671

Optimize the time for tests so that cucumber:boot can be reenabled on travis-ci and so tests are just faster in general.

Optimizations:
# `msfconsole --defer-module-loads` will turn off module loading and loading the module cache from the database. This mode is only recommended for testing `msfconsole` commands that don't need modules.
# Use `msfconsole --defer-module-loads` in cucumber features
# Use `--execute-command` instead of running `msfconsole` interactively and typing commands.
# Don't use simplecov in the `cucumber` process, only the child `msfconsole` processes
# Disable simplecov on travis-ci as reports are not archived. (Coverage remains available locally for checking test coverage unless `bundle install --without coverage` is used.)
# Don't install development gems on travis-ci
# Don't install gem documentation on travis-ci
# Combine cucumber and cucumber boot into same travis-ci job to amortize setup costs

# Verification Steps

## Methodology
- [ ] Review [data and notes](https://docs.google.com/spreadsheets/d/1vCRi_38Go3Wfq670eDCXMTSSXL1blSbk6NQsgpOqfYw/edit?usp=sharing)
- [ ] VERIFY no errors in methods or data that make optimizations invalid.

## Travis-CI build is faster
- [ ] Look at [build time of this PR](https://travis-ci.org/limhoff-r7/metasploit-framework/builds/43352627)
- [ ] Look at [build time of master](https://travis-ci.org/rapid7/metasploit-framework/builds/43288489)
- [ ] VERIFY this PR is faster.

## Travis-CI setup

### job consolidation
- [ ] Look at [job list](https://travis-ci.org/limhoff-r7/metasploit-framework/builds/43352627)
- [ ] VERIFY `cucumber` and `cucumber:boot` are listed together in ENV
- [ ] VERIFY both `cucumber` and `cucumber:boot` are run in the [job](https://travis-ci.org/limhoff-r7/metasploit-framework/jobs/43352629)

### install changes
- [ ] Look at [build log](https://travis-ci.org/limhoff-r7/metasploit-framework/jobs/43352628)
- [ ] VERIFY `development` group is not installed: `redcarpet`, `yard`, and `pry` should not be in `install.bundler` section of log.
- [ ] VERIFY `coverage` group is not installed: `simplecov` should not be in `install.bundler` section of log.