rspec/rspec-core — pull request #1292 by myronmarston (2014-02-07 02:45:37+00:00)
Title: Add some regression tests for some popular custom formatter gems.

For #1291.

So currently fuubar, nyancat and fivemat are all broken against the latest formatter changes:

```
➜ rspec-core git:(add-formatter-regression-tests) ✗ be cucumber --profile wip
Using the wip profile...
@wip @announce
Feature: Regression tests for legacy custom formatters

Background: # features/formatters/regression_tests.feature:4
Given a file named "spec/passing_and_failing_spec.rb" with: # aruba-0.5.4/lib/aruba/cucumber.rb:24
$ export SPEC_OPTS="-r/Users/myron/code/rspec-dev/repos/rspec-core/features/support/require_expect_syntax_in_aruba_specs.rb"
"""
RSpec.describe "Some examples" do
it "passes" do
expect(1).to eq(1)
end

it "fails" do
expect(1).to eq(2)
end

context "nested" do
it "passes" do
expect(1).to eq(1)
end

it "fails" do
expect(1).to eq(2)
end
end
end
"""
And a file named "spec/bar_spec.rb" with: # aruba-0.5.4/lib/aruba/cucumber.rb:24
"""
RSpec.describe "Some pending examples" do
context "pending" do
it "is reported as pending" do
pending { expect(1).to eq(2) }
end

it "is reported as failing" do
pending { expect(1).to eq(1) }
end
end

context "skip" do
it "does not run the example" do
skip
end
end
end
"""

Scenario: Use fuubar formatter # features/formatters/regression_tests.feature:48
When I run `rspec --format Fuubar` # aruba-0.5.4/lib/aruba/cucumber.rb:84
$ cd /Users/myron/code/rspec-dev/repos/rspec-core/tmp/aruba
$ rspec --format Fuubar

/Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/fuubar-1.3.2/lib/fuubar.rb:18:in `initialize': undefined local variable or method `example_count' for # (NameError)
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters.rb:99:in `new'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters.rb:99:in `add'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration.rb:583:in `add_formatter'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:92:in `block in load_formatters_into'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:92:in `each'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:92:in `load_formatters_into'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:18:in `configure'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:20:in `run'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:100:in `run'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:31:in `invoke'
from /Users/myron/code/rspec-dev/repos/rspec-core/exe/rspec:4:in `'
from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `load'
from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `

'
Then the output should contain "TBD" # aruba-0.5.4/lib/aruba/cucumber.rb:131
expected "/Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/fuubar-1.3.2/lib/fuubar.rb:18:in `initialize': undefined local variable or method `example_count' for # (NameError)\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters.rb:99:in `new'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters.rb:99:in `add'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration.rb:583:in `add_formatter'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:92:in `block in load_formatters_into'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:92:in `each'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:92:in `load_formatters_into'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:18:in `configure'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:20:in `run'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:100:in `run'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:31:in `invoke'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/exe/rspec:4:in `'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `load'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `

'\n" to include "TBD"
Diff:
@@ -1,2 +1,15 @@
-TBD
+/Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/fuubar-1.3.2/lib/fuubar.rb:18:in `initialize': undefined local variable or method `example_count' for # (NameError)
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters.rb:99:in `new'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters.rb:99:in `add'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration.rb:583:in `add_formatter'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:92:in `block in load_formatters_into'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:92:in `each'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:92:in `load_formatters_into'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:18:in `configure'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:20:in `run'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:100:in `run'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:31:in `invoke'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/exe/rspec:4:in `'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `load'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `

'
(RSpec::Expectations::ExpectationNotMetError)
features/formatters/regression_tests.feature:50:in `Then the output should contain "TBD"'

Scenario: Use fivemat formatter # features/formatters/regression_tests.feature:52
When I run `rspec --format Fivemat` # aruba-0.5.4/lib/aruba/cucumber.rb:84
$ export SPEC_OPTS="-r/Users/myron/code/rspec-dev/repos/rspec-core/features/support/require_expect_syntax_in_aruba_specs.rb"
$ cd /Users/myron/code/rspec-dev/repos/rspec-core/tmp/aruba
$ rspec --format Fivemat
Some pending examples **FFFF
/Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/fivemat-1.2.1/lib/fivemat/rspec.rb:63:in `instance_method': undefined method `start_dump' for class `RSpec::Core::Formatters::BaseTextFormatter' (NameError)
from /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/fivemat-1.2.1/lib/fivemat/rspec.rb:63:in `start_dump'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters/legacy_formatter.rb:177:in `start_dump'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:114:in `block in notify'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `each'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `notify'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:96:in `finish'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `ensure in report'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `report'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:24:in `run'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:100:in `run'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:31:in `invoke'
from /Users/myron/code/rspec-dev/repos/rspec-core/exe/rspec:4:in `'
from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `load'
from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `

'
Then the output should contain "TBD" # aruba-0.5.4/lib/aruba/cucumber.rb:131
expected "Some pending examples **FFFF\n\n/Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/fivemat-1.2.1/lib/fivemat/rspec.rb:63:in `instance_method': undefined method `start_dump' for class `RSpec::Core::Formatters::BaseTextFormatter' (NameError)\n\tfrom /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/fivemat-1.2.1/lib/fivemat/rspec.rb:63:in `start_dump'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters/legacy_formatter.rb:177:in `start_dump'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:114:in `block in notify'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `each'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `notify'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:96:in `finish'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `ensure in report'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `report'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:24:in `run'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:100:in `run'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:31:in `invoke'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/exe/rspec:4:in `'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `load'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `

'\n" to include "TBD"
Diff:
@@ -1,2 +1,18 @@
-TBD
+Some pending examples **FFFF
+
+/Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/fivemat-1.2.1/lib/fivemat/rspec.rb:63:in `instance_method': undefined method `start_dump' for class `RSpec::Core::Formatters::BaseTextFormatter' (NameError)
+	from /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/fivemat-1.2.1/lib/fivemat/rspec.rb:63:in `start_dump'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters/legacy_formatter.rb:177:in `start_dump'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:114:in `block in notify'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `each'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `notify'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:96:in `finish'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `ensure in report'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `report'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:24:in `run'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:100:in `run'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:31:in `invoke'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/exe/rspec:4:in `'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `load'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `

'
(RSpec::Expectations::ExpectationNotMetError)
features/formatters/regression_tests.feature:54:in `Then the output should contain "TBD"'

Scenario: Use nyancat formatter # features/formatters/regression_tests.feature:56
When I run `rspec --format NyanCatFormatter` # aruba-0.5.4/lib/aruba/cucumber.rb:84
$ export SPEC_OPTS="-r/Users/myron/code/rspec-dev/repos/rspec-core/features/support/require_expect_syntax_in_aruba_specs.rb"
$ cd /Users/myron/code/rspec-dev/repos/rspec-core/tmp/aruba
$ rspec --format NyanCatFormatter
*F
An error occurred
NoMethodError: undefined method `-' for nil:NilClass
occurred at /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/nyan-cat-formatter-0.5.2/lib/nyan_cat_formatter.rb:97:in `terminal_width'

An error occurred
NoMethodError: undefined method `-' for nil:NilClass
occurred at /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/nyan-cat-formatter-0.5.2/lib/nyan_cat_formatter.rb:97:in `terminal_width'

F
stty: stdin isn't a terminal
stty: stdin isn't a terminal
stty: stdin isn't a terminal
/Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/nyan-cat-formatter-0.5.2/lib/nyan_cat_formatter/rspec2.rb:26:in `start_dump': wrong number of arguments (1 for 0) (ArgumentError)
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters/legacy_formatter.rb:67:in `start_dump'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters/legacy_formatter.rb:177:in `start_dump'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:114:in `block in notify'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `each'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `notify'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:96:in `finish'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `ensure in report'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `report'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:24:in `run'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:100:in `run'
from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:31:in `invoke'
from /Users/myron/code/rspec-dev/repos/rspec-core/exe/rspec:4:in `'
from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `load'
from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `

'
Then the output should contain "TBD" # aruba-0.5.4/lib/aruba/cucumber.rb:131
expected "*F\nAn error occurred \n NoMethodError: undefined method `-' for nil:NilClass\n occurred at /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/nyan-cat-formatter-0.5.2/lib/nyan_cat_formatter.rb:97:in `terminal_width'\n\n\nAn error occurred \n NoMethodError: undefined method `-' for nil:NilClass\n occurred at /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/nyan-cat-formatter-0.5.2/lib/nyan_cat_formatter.rb:97:in `terminal_width'\n\nF\nstty: stdin isn't a terminal\nstty: stdin isn't a terminal\nstty: stdin isn't a terminal\n/Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/nyan-cat-formatter-0.5.2/lib/nyan_cat_formatter/rspec2.rb:26:in `start_dump': wrong number of arguments (1 for 0) (ArgumentError)\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters/legacy_formatter.rb:67:in `start_dump'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters/legacy_formatter.rb:177:in `start_dump'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:114:in `block in notify'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `each'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `notify'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:96:in `finish'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `ensure in report'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `report'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:24:in `run'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:100:in `run'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:31:in `invoke'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/exe/rspec:4:in `'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `load'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `

'\n" to include "TBD"
Diff:
@@ -1,2 +1,30 @@
-TBD
+*F
+An error occurred
+ NoMethodError: undefined method `-' for nil:NilClass
+ occurred at /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/nyan-cat-formatter-0.5.2/lib/nyan_cat_formatter.rb:97:in `terminal_width'
+
+
+An error occurred
+ NoMethodError: undefined method `-' for nil:NilClass
+ occurred at /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/nyan-cat-formatter-0.5.2/lib/nyan_cat_formatter.rb:97:in `terminal_width'
+
+F
+stty: stdin isn't a terminal
+stty: stdin isn't a terminal
+stty: stdin isn't a terminal
+/Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/nyan-cat-formatter-0.5.2/lib/nyan_cat_formatter/rspec2.rb:26:in `start_dump': wrong number of arguments (1 for 0) (ArgumentError)
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters/legacy_formatter.rb:67:in `start_dump'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters/legacy_formatter.rb:177:in `start_dump'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:114:in `block in notify'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `each'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `notify'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:96:in `finish'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `ensure in report'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `report'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:24:in `run'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:100:in `run'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:31:in `invoke'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/exe/rspec:4:in `'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `load'
+	from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `

'
(RSpec::Expectations::ExpectationNotMetError)
features/formatters/regression_tests.feature:58:in `Then the output should contain "TBD"'

Failing Scenarios:
cucumber -p wip features/formatters/regression_tests.feature:48 # Scenario: Use fuubar formatter
cucumber -p wip features/formatters/regression_tests.feature:52 # Scenario: Use fivemat formatter
cucumber -p wip features/formatters/regression_tests.feature:56 # Scenario: Use nyancat formatter

3 scenarios (3 failed)
12 steps (3 failed, 9 passed)
0m1.988s

The --wip switch was used, so the failures were expected. All is good.
```

@JonRowe -- do you want to take a stab at fixing these or would you like me to?