hotsh/rstat.us — pull request #745 by wilkie (2013-02-19 12:13:55+00:00)
Title: Bugfix/faster tests through rakefile

Ensures that no environment is loaded when running tests. Edits the Rakefile to provide a clean path to the tester. Saves 8 seconds off of any test execution. Benchmarks follow:

## Small single file

### Original

```
$ time rake test:file[test/services/feed_service_test.rb]
Run options: --seed 321

Finished tests in 0.200419s, 24.9478 tests/s, 24.9478 assertions/s.

real 0m19.105s
user 0m18.263s
sys 0m0.790s

```

### Refactoring through #743

```
$ time rake test:file[test/services/feed_service_test.rb]
Run options: --seed 45013

Finished tests in 0.015600s, 320.5111 tests/s, 384.6133 assertions/s.

real 0m18.919s
user 0m18.087s
sys 0m0.780s

```

### Not requiring test_helper.rb plus #743

```
$ time rake test:file[test/services/feed_service_test.rb]
Run options: --seed 55063

Finished tests in 0.018919s, 264.2831 tests/s, 317.1397 assertions/s.

real 0m7.612s
user 0m7.283s
sys 0m0.310s

```

### Rakefile edit

```
$ time rake test:file[test/services/feed_service_test.rb]
Run options: --seed 4084

Finished tests in 0.192015s, 26.0396 tests/s, 26.0396 assertions/s.

real 0m13.249s
user 0m12.833s
sys 0m0.380s

```

### Rakefile edit and not requiring test_helper.rb plus #743

```
$ time rake test:file[test/services/feed_service_test.rb]
Run options: --seed 47619

Finished tests in 0.018980s, 263.4376 tests/s, 316.1251 assertions/s.

real 0m1.758s
user 0m1.640s
sys 0m0.110s

```
This represents the savings when all effort is applied to quicken tests.

Nearly all overhead is gone. Conclusion: We should strive to remove database dependencies throughout the unit tests and we can eliminate all but rake's IO overhead.

## All units [test:models]

### Original

```
$ time rake test:models
Run options: --seed 33656

Finished tests in 27.609755s, 5.8675 tests/s, 8.8012 assertions/s.

real 0m46.603s
user 0m44.830s
sys 0m0.907s

```

### Rakefile edit (without refactoring unittests themselves)

```
$ time rake test:models
Run options: --seed 64078

Finished tests in 25.342149s, 6.3925 tests/s, 9.5888 assertions/s.

real 0m38.577s
user 0m37.417s
sys 0m0.653s
```

8 seconds for an easy fix.

## Everything [test:all]

### Original

```
$ time rake test:all
Run options: --seed 60752

Finished tests in 256.068263s, 1.8120 tests/s, 3.4288 assertions/s.

real 4m35.657s
user 3m40.093s
sys 0m2.680s

```

### Rakefile edit (no refactoring of tests)

```
$ time rake test:all
Run options: --seed 12314

Finished tests in 253.621540s, 1.8295 tests/s, 3.4619 assertions/s.

real 4m27.334s
user 3m31.483s
sys 0m2.433s
```

Looks like it is only a one time 7-8s save. But that makes sense.