authmillenon/pycomicvine — pull request #1 by xchewtoyx (2013-06-16 06:18:25+00:00)
Title: Paging update to follow new API behaviour and work around an API bug

The comicvine API now ignores the offset in a request, and instead uses the page parameter for large result sets. The current pycomicvine behaviour results in large numbers of duplicate results, as well as missing results.

This proposed code does two things - it adds the page parameter to the request, in order to get the additional results as expected, it also includes code to work around a bug in the API. Sometimes you will ask for 100 results, but only get 99. I tried a couple of approaches to this, but in the end just settled on returning None for missing entries. I played with a couple of other ideas for handling this, but this seemed easiest.

As examples of this in practice, when I use the code from HEAD to get a list of volumes containing "Batman" I get 742 results. When I de-duplicate these I am only left with 100. With the new code I get 742 unique results:

rgh@eduardo ~/git/pycomicvine $ python volumelist.py batman | sort -u | wc -l
100

rgh@eduardo ~/git/mycomicvine $ python volumelist.py batman | sort -u | wc -l
742

With a similar query for "Spider-man" (which shows the buggy behaviour described above) there is a similar discrepancy with the results. The search reports 632 issues, yet it can be seen that there are 100 unique results from the code at HEAD, and 630 from the revised code.

rgh@eduardo ~/git/pycomicvine $ python volumelist.py spider-man | sort -u | wc -l
100

rgh@eduardo ~/git/mycomicvine $ python volumelist.py spider-man | sort -u | wc -l
630

This patch works well for my calibre plugin, hopefully this will be useful for the base library too...