Change method comment to more accurately describe its behavior.
Do not stub FeedClient.fetch in all ScheduledUpdateFeedJob tests.

It is not really necessary to stub it in all tests; RestClient.get is stubbed in spec_helper.rb, which ensures that no HTTP calls are made during testing. Those tests in scheduled_update_feed_job_spec.rb that need to stub FeedClient.fetch do it explicitly as part of the test.

Also, I will need to invoke the actual implementation of FeedClient.fetch to test a new feature: if a scheduled feed update fails, the job will attempt to perform autodiscovery, in case the feed URL has changed but the website URL remains the same (which in my experience happens quite often).
When saving a Feed instance, the url attribute value defaults to the fetch_url value if no value has been specified.

The fetch_url attribute is required to have a value, so the url attribute is now guaranteed to have a value.

This will help with certain upcoming changes (attempting autodiscovery when a scheduled feed update fails).
If FeedClient.fetch is invoked with the perform_autodiscovery argument set to true, feed.url will be passed to RestClient.get instead of feed.fetch_url. Also no HTTP caching headers will be sent.

This is intended for the case in which a scheduled update of the feed fails. In this case we want to attempt autodiscovery using feed.url; attempting it on feed.fetch_url is useless because that URL is failing. feed.url should lead to the webpage, which is our best bet to get the updated fetch_url if it has changed.

Also when we are performing autodiscovery we don't want to receive a "304 not modified" response, we want the last version of the actual HTML document to be able to do autodiscovery. So now the method does not send any HTTP caching headers if it's been told to perform autodiscovery.
Fixed bug (referencing the "headers" hash variable before it was defined).
When a scheduled feed update fails, attempt to perform autodiscovery on the website URL.

This is done only if the update failure raises an error that could mean that the URL is unavailable (timeout, host not found, etc). Other kind of errors that could arise e.g. from bugs in the app or problems in the server do not result in an autodiscovery attempt.

If a different fetch_url is discovered from the website, the fetch_url attribute of the feed is updated with the new value and a fetch is attempted from this url, which could result in new entries and in an updated feed title.

This is intended for the case in which a feed owner decides to change its URL, e.g. because he migrates from an in-house solution to feedburner. In these cases usually the website URL does not change, and often the site owner does not even give a warning that the old feed URL is no longer valid. Hopefully he will remember to update the autodiscovery information in his website with the new feed URL.

The old behavior was: the scheduled feed updates started failing, and after 1 week the feed would be marked as permenently unavailable. The feedbunch admin had this week to become aware of the problem and fix it manually by going to the website, copying the new feed URL and updating the feed with it using ActiveAdmin. It was a less than ideal solution that required too much manual intervention.

With the new behavior, feedbunch will try to autodetect the new feed url when this happens. As long as the feed owner updates the autodiscovery information in his website in the week following the URL change, feedbunch will update the feed data and mark the feed as "not failing" automatically, without manual intervention.

Of course if the site owner does not update the autodiscovery data or does not even support autodiscovery, this will fail and manual intervention will still be required. I'd like to send people who do this on a one-way trip to a special circle of hell without internet access, but alas, I have to suffer living in an imperfect world.