Repository: AnalogJ/lexicon

Issue:
Cloudflare provider only lists first DNS 100 entries for domain
How to reproduce:

* Have a CloudFlare domain with > 100 entries
* Create a new entry starting with a "high" letter, e.g. `zzzz.<domain>`
* List entries: only the first 100 entries will be returned, making it appear as though `zzzz` had not been created.


```
    def _list_records(self, rtype=None, name=None, content=None):
        filter_obj = {'per_page': 100}      ####### NOTE
        if rtype:
            filter_obj['type'] = rtype
        if name:
            filter_obj['name'] = self._full_name(name)
        if content:
            filter_obj['content'] = content

        payload = self._get(
            '/zones/{0}/dns_records'.format(self.domain_id), filter_obj)   
           ### subsequent pages not retrieved

        records = []
        for record in payload['result']:
            processed_record = {
                'type': record['type'],
                'name': record['name'],
                'ttl': record['ttl'],
                'content': record['content'],
                'id': record['id']
            }
            records.append(processed_record)

        LOGGER.debug('list_records: %s', records)
        return records
```

This causes weird errors whereby an existing entry is not returned if falls after the 100th entry.

Without specifying the "per page" parameter, only the first 20 entries are returned.

Besides a quick-n-dirty solution of raising that number to e.g. 1000, the proper solution would be to implement proper pagination support for the _get method.


Discussion:
The payload conatins two entries:
* `result`
* `result_info`:
```
'result_info': {'page': 1, 'per_page': 2, 'total_pages': 81, 'count': 2, 'total_count': 161}
```

Resolving patch:
diff --git a/lexicon/providers/cloudflare.py b/lexicon/providers/cloudflare.py
--- a/lexicon/providers/cloudflare.py
+++ b/lexicon/providers/cloudflare.py
@@ -74,21 +74,31 @@ def _list_records(self, rtype=None, name=None, content=None):
         if content:
             filter_obj['content'] = content
 
-        payload = self._get(
-            '/zones/{0}/dns_records'.format(self.domain_id), filter_obj)
-
         records = []
-        for record in payload['result']:
-            processed_record = {
-                'type': record['type'],
-                'name': record['name'],
-                'ttl': record['ttl'],
-                'content': record['content'],
-                'id': record['id']
-            }
-            records.append(processed_record)
+        while True:
+            payload = self._get(
+                '/zones/{0}/dns_records'.format(self.domain_id), filter_obj)
+
+            LOGGER.debug("payload: %s", payload)
+
+            for record in payload['result']:
+                processed_record = {
+                    'type': record['type'],
+                    'name': record['name'],
+                    'ttl': record['ttl'],
+                    'content': record['content'],
+                    'id': record['id']
+                }
+                records.append(processed_record)
+
+            pages = payload['result_info']['total_pages']
+            page = payload['result_info']['page']
+            if page >= pages:
+                break
+            filter_obj['page'] = page + 1
 
         LOGGER.debug('list_records: %s', records)
+        LOGGER.debug('Number of records retrieved: %d', len(records))
         return records
 
     # Create or update a record.


Test patch: