Repository: ansible/ansible

Issue:
ec2.py return a maximum of 100 rds instances from AWS
##### Issue Type:

<!--- Please pick one and delete the rest: -->
- Bug Report
##### Ansible Version:

```
ansible 2.0.1.0
```
##### Ansible Configuration:

None
##### Environment:

Debian
##### Summary:

When using the script `ec2.py` to pull information from AWS, your list of RDS instances would be limited to 100 results only.
This is due to module RDS in boto:
http://boto.cloudhackers.com/en/latest/ref/rds.html#module-boto.rds
Method name: `get_all_dbinstances`
##### Steps To Reproduce:

```
ec2.py --list | grep rds
```

Inside the above script, you'll find the following:

```
    def get_rds_instances_by_region(self, region):
        ''' Makes an AWS API call to the list of RDS instances in a particular
        region '''

        try:
            conn = self.connect_to_aws(rds, region)
            if conn:
                instances = conn.get_all_dbinstances()
                for instance in instances:
                    self.add_rds_instance(instance, region)
```

<!--- You can also paste gist.github.com links for larger files -->
##### Expected Results:

Receive a list of all RDS instances present on AWS
##### Actual Results:

Limitation of 100 RDS instances

ec2.py return a maximum of 100 rds instances from AWS
##### Issue Type:

<!--- Please pick one and delete the rest: -->
- Bug Report
##### Ansible Version:

```
ansible 2.0.1.0
```
##### Ansible Configuration:

None
##### Environment:

Debian
##### Summary:

When using the script `ec2.py` to pull information from AWS, your list of RDS instances would be limited to 100 results only.
This is due to module RDS in boto:
http://boto.cloudhackers.com/en/latest/ref/rds.html#module-boto.rds
Method name: `get_all_dbinstances`
##### Steps To Reproduce:

```
ec2.py --list | grep rds
```

Inside the above script, you'll find the following:

```
    def get_rds_instances_by_region(self, region):
        ''' Makes an AWS API call to the list of RDS instances in a particular
        region '''

        try:
            conn = self.connect_to_aws(rds, region)
            if conn:
                instances = conn.get_all_dbinstances()
                for instance in instances:
                    self.add_rds_instance(instance, region)
```

<!--- You can also paste gist.github.com links for larger files -->
##### Expected Results:

Receive a list of all RDS instances present on AWS
##### Actual Results:

Limitation of 100 RDS instances



Discussion:
Looks like `get_all_dbinstances` by default returns only 100 records and uses pagination if you want more records, so there's an extra iterative operation needed in order to get all results.

i need to find the ticket, already added paging for another module, we should just add a generic function for all aws objects as they seem to have the same paging interface

please test the above PR

Looks like `get_all_dbinstances` by default returns only 100 records and uses pagination if you want more records, so there's an extra iterative operation needed in order to get all results.

i need to find the ticket, already added paging for another module, we should just add a generic function for all aws objects as they seem to have the same paging interface

please test the above PR


Resolving patch:
diff --git a/contrib/inventory/ec2.py b/contrib/inventory/ec2.py
--- a/contrib/inventory/ec2.py
+++ b/contrib/inventory/ec2.py
@@ -498,9 +498,14 @@ def get_rds_instances_by_region(self, region):
         try:
             conn = self.connect_to_aws(rds, region)
             if conn:
-                instances = conn.get_all_dbinstances()
-                for instance in instances:
-                    self.add_rds_instance(instance, region)
+                marker = None
+                while True:
+                    instances = conn.get_all_dbinstances(marker=marker)
+                    marker = instances.marker
+                    for instance in instances:
+                        self.add_rds_instance(instance, region)
+                    if not marker:
+                        break
         except boto.exception.BotoServerError as e:
             error = e.reason
 


Test patch: