rspec/rspec-core — Issue #556: metadata Procs are called even if the tag is not present and rescued
Author: eregon
Created: 2012-01-17 11:28:54+00:00
State: open

Hello,
I did a quick search so I hope this is no duplicate.

When you define a filter with a Proc, such as:

```ruby
RSpec.configure do |config|
config.filter_run_excluding :fails_on => lambda { |implementations|
implementations.include? some_ruby
}
end
```

The Proc is called for every example (with `nil` when the example has no :fails_on key in its metadata).
This is silenced to the user by the (in my opinion) bad use of the `rescue false` in `rspec/core/matadata.rb` method `filter_applies?` lines 180-190:

```ruby
case value
...
when Proc
if value.arity == 2
# Pass the metadata hash to allow the proc to check if it even has the key.
# This is necessary for the implicit :if exclusion filter:
# { } # => run the example
# { :if => nil } # => exclude the example
# The value of metadata[:if] is the same in these two cases but
# they need to be treated differently.
value.call(metadata[key], metadata) rescue false
else
value.call(metadata[key]) rescue false
end
...
```

I somewhat understand the issue with :if, but isn't a check such as `key == :if or key == :unless` enough?

So the NameError that should be produced by calling `nil.include? some_ruby` is eaten silently.
It means the filter should actually be written in such a way to handle nil:

```ruby
config.filter_run_excluding :fails_on => lambda { |implementations|
implementations and implementations.include? some_ruby
}
```

... which seems very unintuitive for me and might produce inverted filtering.

I believe the Proc should not be called when the example does not have the tag key.

I'm still confused with RSpec tags, partly due to unexpected semantics, so please excuse me if I'm just wrong, but eating errors is never a good idea IMHO (I would much prefer to get the error).

Could someone explain me the rationale behind this?