Repository: ansible/ansible-modules-core

Issue:
ec2_elb only looks at first 400 ELBs
**Issue Type: Bug Report**

**Ansible Version**: Ansible 1.9.3
**Boto Version**: 2.38.0
**Python Version**: 2.7.5
**Environment**: server OSX Mavericks, destination RHEL 6.6

**Summary**:
The ec2_elb module uses the Boto call "get_all_load_balancers" to dump all ELBs in an account.  However it only looks at the first 400 ELBs.

I verified this by adding debugging to the module to see what it gets back from the Boto call, and it is always exactly 400 ELBs.  However I can see all of our ELBs (currently at 488) when I use the AWS CLI or GUI.

The problem is the ec2_elb module does not make use of the "marker" that the Boto call returns if there are more than 400 ELBs.  The Boto 2.3.8 docs describe the use of an optional "marker" to retrieve the next 400 ELBs:

```
get_all_load_balancers(load_balancer_names=None, marker=None)

marker (string) – Use this only when paginating results and only in follow-up request after you’ve
received a response where the results are truncated. Set this to the value of the Marker element in
the response you just received.
```

**Steps to Reproduce**:
First, you need an account with > 400 ELBs.  Probably rare, but we have one...

Example task:

```
- name: "Register EC2 instance with new ELB"
  local_action:
    module: ec2_elb
    instance_id: my_instance_id
    ec2_elbs: my_test_elb
    state: 'present'
    region: us-east-1
```

Output:

```
TASK: [channel_switch | Register EC2 instance with new ELB] ****************** 
failed: [localhost -> 127.0.0.1] => (item=my_instance_id) => {"failed": true, "item": "my_instance_id"}
msg: ELB my_test_elb does not exist
```



Discussion:
cc @jarv

Just checking if there is any resolution for this. Or any work around to get the instance added to ELB(using ansible module ec2_elb) when >400 ELB's are present. I am having the same issue, was able to add using AWS CLI.

For now, i am going with this following work around, executing AWS CLI from my play.
- name: Add EC2 instance to ELB {{ elb_result.elb.name }} using AWS - CLI from within ansible play
  command: "sudo -E aws elb register-instances-with-load-balancer --load-balancer-name {{ elb_result.elb.name }} --instances i-456r3546 --profile <<MyProfileHereIfNeeded>>"
  environment:
    http_proxy: http://{{ proxyUserId }}:{{ proxyPwd }}@proxy.com:port
    https_proxy: http://{{ proxyUserId }}:{{ proxyPwd }}@proxy.com:port

can you test out my patch? I don't have 400+ elbs to look at


Resolving patch:
diff --git a/cloud/amazon/ec2_elb.py b/cloud/amazon/ec2_elb.py
--- a/cloud/amazon/ec2_elb.py
+++ b/cloud/amazon/ec2_elb.py
@@ -50,10 +50,10 @@
     choices: [ "yes", "no" ]
   wait:
     description:
-      - Wait for instance registration or deregistration to complete successfully before returning.  
+      - Wait for instance registration or deregistration to complete successfully before returning.
     required: false
     default: yes
-    choices: [ "yes", "no" ] 
+    choices: [ "yes", "no" ]
   validate_certs:
     description:
       - When set to "no", SSL certificates will not be validated for boto versions >= 2.6.0.
@@ -87,7 +87,7 @@
   - myrole
 post_tasks:
   - name: Instance Register
-    local_action: 
+    local_action:
       module: ec2_elb
       instance_id: "{{ ansible_ec2_instance_id }}"
       ec2_elbs: "{{ item }}"
@@ -256,12 +256,23 @@ def _get_instance_lbs(self, ec2_elbs=None):
            ec2_elbs = self._get_auto_scaling_group_lbs()
 
         try:
-            elb = connect_to_aws(boto.ec2.elb, self.region, 
-                                 **self.aws_connect_params)
+            elb = connect_to_aws(boto.ec2.elb, self.region, **self.aws_connect_params)
         except (boto.exception.NoAuthHandlerFound, StandardError), e:
             self.module.fail_json(msg=str(e))
 
-        elbs = elb.get_all_load_balancers()
+        elbs = []
+        marker = None
+        while True:
+            try:
+                newelbs = elb.get_all_load_balancers(marker=marker)
+                marker = newelbs.next_marker
+                if not marker:
+                    break
+                elbs.extend(newelbs)
+            except TypeError:
+                # Older version of boto do not allow for params
+                elbs = elb.get_all_load_balancers()
+                break
 
         if ec2_elbs:
             lbs = sorted(lb for lb in elbs if lb.name in ec2_elbs)
@@ -302,8 +313,7 @@ def _get_auto_scaling_group_lbs(self):
     def _get_instance(self):
         """Returns a boto.ec2.InstanceObject for self.instance_id"""
         try:
-            ec2 = connect_to_aws(boto.ec2, self.region, 
-                                 **self.aws_connect_params)
+            ec2 = connect_to_aws(boto.ec2, self.region, **self.aws_connect_params)
         except (boto.exception.NoAuthHandlerFound, StandardError), e:
             self.module.fail_json(msg=str(e))
         return ec2.get_only_instances(instance_ids=[self.instance_id])[0]
@@ -330,7 +340,7 @@ def main():
 
     region, ec2_url, aws_connect_params = get_aws_connection_info(module)
 
-    if not region: 
+    if not region:
         module.fail_json(msg="Region must be specified as a parameter, in EC2_REGION or AWS_REGION environment variables or in boto configuration file")
 
     ec2_elbs = module.params['ec2_elbs']
@@ -342,8 +352,7 @@ def main():
         module.fail_json(msg="ELBs are required for registration")
 
     instance_id = module.params['instance_id']
-    elb_man = ElbManager(module, instance_id, ec2_elbs, 
-                         region=region, **aws_connect_params)
+    elb_man = ElbManager(module, instance_id, ec2_elbs, region=region, **aws_connect_params)
 
     if ec2_elbs is not None:
         for elb in ec2_elbs:


Test patch: