NzbDrone/NzbDrone — pull request #112 by Taloth (2014-09-07 12:15:42+00:00)
Title: Indexer Refactor

**Work in Progress**

Transplanted the indexer refactor code from torrents on dev so we can develop and integrate this.

@kayone I've included as separate commits the changes made to the httpprovider stuff on the torrents branch.
DownloadXml was added to allow checking whether the indexer returned xml content, this to be able to report to the user that the indexer was down/blocked.
I would appreciate it if you can integrate this somehow in your revision of the HttpProvider code.

@markus101 @kayone
Obviously this isn't finished at all. But I wanted to get a PR out so we can have some feedback.
The branch also contains my preliminary refactoring I did earlier so it might seem a bit messy.
But we're going another route after I was able to bounch a couple of ideas back and forth to kayone.
I've only refactored Newznab and Wombles to use the ```RequestGenerator``` and derived them from RssIndexerBase.
The other indexers are derived from ```LegacyRssIndexerBase``` and basically do the 'old' thing. So ignore any code changes related to ```LegacyRssIndexerBase``` and it's derivatives.

Atm we have the following design in mind.
```IIndexer``` is the entry point for searching that indexer. So no more asking the ```FetchFeedService``` to do that. That way we allow the indexer to determine how the search should happen.

The ```IndexerBase``` (currently ```RssIndexerBase```) is essentially a composite. It contains a ```IProcessIndexerResponse GetProcessor``` method to return a processor that parses the results from the indexer. By being based on IndexerResponse instead of Xml we get a lot more flexibility.
This replaces the ```GetParser``` method.

We also extracted the *Urls methods into a RequestGenerator.
The ```IIndexerRequestGenerator``` returns, as you guess, ```IndexerRequest``` instances. Which could be Urls, json etc.

It uses a ```IEnumerable``` for paging, which basically is a yield return loop that returns the 'url' for subsequent pages.

It can also return multiple different queries since the request generator returns ```IList>```. For example for multiple scene titles, while still supporting paging for each individual query.

Currently the ```RssIndexerBase``` does the heavy lifting of looping over those and combining the results.
To accomplish that I current call the IHttpProvider implementation.

**TL;DR:** Please look at the ```IIndexerRequestGenerator```and ```IProcessIndexerResponse``` implementation for Newznab and Womble and tell me if you like it.