you should do this:
  page.should have_selector(selector, visible: false)
  page.should_not have_selector(selector)

that way you test that the element is not visible in the page ;)