Removing duplicates from reservation's groupSet

Fog::Compute::AWS::Server objects produced from describe_instances contain duplicates in the group's array.

As a result, tools built on Fog e.g <a href="https://github.com/opscode/knife-ec2">knife-ec2</a> produce confusing output:

<pre>
# knife ec2 server list
Instance ID  Name        Zone        Public IP  Private IP  Flavor    Image         SSH Key  Sec Group                                   State  
i-XXXXXXXX   mastertest  us-west-2a                         t1.micro  ami-XXXXXXXX  artem    [webservers, webservers]                    stopped
i-XXXXXXXX   grouptest   us-west-2a                         t1.micro  ami-XXXXXXXX  artem    [webservers, default, webservers, default]  stopped
</pre>

This patch fixes the describe_instances parser and eliminates duplicates.


Here are the details.

A sample AWS response to describe_instances request:
<pre>
&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;DescribeInstancesResponse xmlns="http://ec2.amazonaws.com/doc/2011-11-01/"&gt;
    &lt;requestId&gt;51b74328-d2d7-4b02-867d-818fec883f7d&lt;/requestId&gt;
    &lt;reservationSet&gt;
        &lt;item&gt;
            &lt;reservationId&gt;XXXXXX&lt;/reservationId&gt;
            &lt;ownerId&gt;XXXXXX&lt;/ownerId&gt;
 --&gt;        &lt;groupSet&gt;  &lt;-- reservation groupSet
                &lt;item&gt;
                    &lt;groupId&gt;XXXXXX&lt;/groupId&gt;
                    &lt;groupName&gt;webservers&lt;/groupName&gt;
                &lt;/item&gt;
            &lt;/groupSet&gt;
            &lt;instancesSet&gt;
                &lt;item&gt;
                    &lt;instanceId&gt;XXXXXX&lt;/instanceId&gt;
                    &lt;imageId&gt;XXXXXX&lt;/imageId&gt;
                    &lt;instanceState&gt;
                        &lt;code&gt;80&lt;/code&gt;
                        &lt;name&gt;stopped&lt;/name&gt;
                    &lt;/instanceState&gt;
                    &lt;privateDnsName/&gt;
                    &lt;dnsName/&gt;
                    &lt;reason&gt;User initiated (2012-01-30 05:44:47 GMT)&lt;/reason&gt;
                    &lt;keyName&gt;artem&lt;/keyName&gt;
                    &lt;amiLaunchIndex&gt;0&lt;/amiLaunchIndex&gt;
                    &lt;productCodes/&gt;
                    &lt;instanceType&gt;t1.micro&lt;/instanceType&gt;
                    &lt;launchTime&gt;2012-01-30T05:00:34.000Z&lt;/launchTime&gt;
                    &lt;placement&gt;
                        &lt;availabilityZone&gt;us-west-2a&lt;/availabilityZone&gt;
                        &lt;groupName/&gt;
                        &lt;tenancy&gt;default&lt;/tenancy&gt;
                    &lt;/placement&gt;
                    &lt;kernelId&gt;aki-ace26f9c&lt;/kernelId&gt;
                    &lt;monitoring&gt;
                        &lt;state&gt;disabled&lt;/state&gt;
                    &lt;/monitoring&gt;
 --&gt;                &lt;groupSet&gt; &lt;-- instance groupSet
                        &lt;item&gt;
                            &lt;groupId&gt;XXXXXX&lt;/groupId&gt;
                            &lt;groupName&gt;webservers&lt;/groupName&gt;
                        &lt;/item&gt;
                    &lt;/groupSet&gt;
                    &lt;stateReason&gt;
                        &lt;code&gt;Client.UserInitiatedShutdown&lt;/code&gt;
                        &lt;message&gt;Client.UserInitiatedShutdown: User initiated shutdown&lt;/message&gt;
                    &lt;/stateReason&gt;
                    &lt;architecture&gt;x86_64&lt;/architecture&gt;
                    &lt;rootDeviceType&gt;ebs&lt;/rootDeviceType&gt;
                    &lt;rootDeviceName&gt;/dev/sda1&lt;/rootDeviceName&gt;
                    &lt;blockDeviceMapping&gt;
                        &lt;item&gt;
                            &lt;deviceName&gt;/dev/sda1&lt;/deviceName&gt;
                            &lt;ebs&gt;
                                &lt;volumeId&gt;XXXXXX&lt;/volumeId&gt;
                                &lt;status&gt;attached&lt;/status&gt;
                                &lt;attachTime&gt;2012-01-30T05:45:08.000Z&lt;/attachTime&gt;
                                &lt;deleteOnTermination&gt;true&lt;/deleteOnTermination&gt;
                            &lt;/ebs&gt;
                        &lt;/item&gt;
                    &lt;/blockDeviceMapping&gt;
                    &lt;virtualizationType&gt;paravirtual&lt;/virtualizationType&gt;
                    &lt;clientToken/&gt;
                    &lt;tagSet&gt;
                        &lt;item&gt;
                            &lt;key&gt;Name&lt;/key&gt;
                            &lt;value&gt;mastertest&lt;/value&gt;
                        &lt;/item&gt;
                    &lt;/tagSet&gt;
                    &lt;hypervisor&gt;xen&lt;/hypervisor&gt;
                &lt;/item&gt;
            &lt;/instancesSet&gt;
        &lt;/item&gt;
        &lt;item&gt;
            &lt;reservationId&gt;XXXXXX&lt;/reservationId&gt;
            &lt;ownerId&gt;XXXXXX&lt;/ownerId&gt;
 --&gt;        &lt;groupSet&gt; &lt;-- reservation groupSet
                &lt;item&gt;
                    &lt;groupId&gt;XXXXXX&lt;/groupId&gt;
                    &lt;groupName&gt;webservers&lt;/groupName&gt;
                &lt;/item&gt;
                &lt;item&gt;
                    &lt;groupId&gt;XXXXXX&lt;/groupId&gt;
                    &lt;groupName&gt;default&lt;/groupName&gt;
                &lt;/item&gt;
            &lt;/groupSet&gt;
            &lt;instancesSet&gt;
                &lt;item&gt;
                    &lt;instanceId&gt;XXXXXX&lt;/instanceId&gt;
                    &lt;imageId&gt;XXXXXX&lt;/imageId&gt;
                    &lt;instanceState&gt;
                        &lt;code&gt;80&lt;/code&gt;
                        &lt;name&gt;stopped&lt;/name&gt;
                    &lt;/instanceState&gt;
                    &lt;privateDnsName/&gt;
                    &lt;dnsName/&gt;
                    &lt;reason&gt;User initiated (2012-01-30 12:01:57 GMT)&lt;/reason&gt;
                    &lt;keyName&gt;artem&lt;/keyName&gt;
                    &lt;amiLaunchIndex&gt;0&lt;/amiLaunchIndex&gt;
                    &lt;productCodes/&gt;
                    &lt;instanceType&gt;t1.micro&lt;/instanceType&gt;
                    &lt;launchTime&gt;2012-01-30T10:44:32.000Z&lt;/launchTime&gt;
                    &lt;placement&gt;
                        &lt;availabilityZone&gt;us-west-2a&lt;/availabilityZone&gt;
                        &lt;groupName/&gt;
                        &lt;tenancy&gt;default&lt;/tenancy&gt;
                    &lt;/placement&gt;
                    &lt;kernelId&gt;aki-ace26f9c&lt;/kernelId&gt;
                    &lt;monitoring&gt;
                        &lt;state&gt;disabled&lt;/state&gt;
                    &lt;/monitoring&gt;
--&gt;                 &lt;groupSet&gt;  &lt;-- instance groupSet
                        &lt;item&gt;
                            &lt;groupId&gt;XXXXXX&lt;/groupId&gt;
                            &lt;groupName&gt;webservers&lt;/groupName&gt;
                        &lt;/item&gt;
                        &lt;item&gt;
                            &lt;groupId&gt;XXXXXX&lt;/groupId&gt;
                            &lt;groupName&gt;default&lt;/groupName&gt;
                        &lt;/item&gt;
                    &lt;/groupSet&gt;
                    &lt;stateReason&gt;
                        &lt;code&gt;Client.UserInitiatedShutdown&lt;/code&gt;
                        &lt;message&gt;Client.UserInitiatedShutdown: User initiated shutdown&lt;/message&gt;
                    &lt;/stateReason&gt;
                    &lt;architecture&gt;x86_64&lt;/architecture&gt;
                    &lt;rootDeviceType&gt;ebs&lt;/rootDeviceType&gt;
                    &lt;rootDeviceName&gt;/dev/sda1&lt;/rootDeviceName&gt;
                    &lt;blockDeviceMapping&gt;
                        &lt;item&gt;
                            &lt;deviceName&gt;/dev/sda1&lt;/deviceName&gt;
                            &lt;ebs&gt;
                                &lt;volumeId&gt;XXXXXX&lt;/volumeId&gt;
                                &lt;status&gt;attached&lt;/status&gt;
                                &lt;attachTime&gt;2012-01-30T12:02:17.000Z&lt;/attachTime&gt;
                                &lt;deleteOnTermination&gt;true&lt;/deleteOnTermination&gt;
                            &lt;/ebs&gt;
                        &lt;/item&gt;
                    &lt;/blockDeviceMapping&gt;
                    &lt;virtualizationType&gt;paravirtual&lt;/virtualizationType&gt;
                    &lt;clientToken/&gt;
                    &lt;tagSet&gt;
                        &lt;item&gt;
                            &lt;key&gt;Name&lt;/key&gt;
                            &lt;value&gt;grouptest&lt;/value&gt;
                        &lt;/item&gt;
                    &lt;/tagSet&gt;
                    &lt;hypervisor&gt;xen&lt;/hypervisor&gt;
                &lt;/item&gt;
            &lt;/instancesSet&gt;
        &lt;/item&gt;
    &lt;/reservationSet&gt;
&lt;/DescribeInstancesResponse&gt;
</pre>

The above request produces the following Fog::Compute::AWS::Server objects:

<pre>
  &lt;Fog::Compute::AWS::Server
    id="XXXXXX",
    ami_launch_index=0,
    availability_zone="us-west-2a",
    block_device_mapping=[{"deviceName"=&gt;"/dev/sda1", "volumeId"=&gt;"XXXXXX", "status"=&gt;"attached", "attachTime"=&gt;2012-01-30 05:45:08 UTC, "deleteOnTermination"=&gt;true}],
    client_token=nil,
    dns_name=nil,
    groups=["XXXXXX", "webservers", "XXXXXX", "webservers"],
    flavor_id="t1.micro",
    image_id="XXXXXX",
    kernel_id="aki-ace26f9c",
    key_name="artem",
    created_at=2012-01-30 05:00:34 UTC,
    monitoring=false,
    placement_group=nil,
    platform=nil,
    product_codes=[],
    private_dns_name=nil,
    private_ip_address=nil,
    public_ip_address=nil,
    ramdisk_id=nil,
    reason="User initiated (2012-01-30 05:44:47 GMT)",
    root_device_name=nil,
    root_device_type="ebs",
    security_group_ids=nil,
    state="stopped",
    state_reason={"code"=&gt;0},
    subnet_id=nil,
    tenancy="default",
    tags={"Name"=&gt;"mastertest"},
    user_data=nil
  &gt;,
   &lt;Fog::Compute::AWS::Server
    id="XXXXXX",
    ami_launch_index=0,
    availability_zone="us-west-2a",
    block_device_mapping=[{"deviceName"=&gt;"/dev/sda1", "volumeId"=&gt;"XXXXXX", "status"=&gt;"attached", "attachTime"=&gt;2012-01-30 12:02:17 UTC, "deleteOnTermination"=&gt;true}],
    client_token=nil,
    dns_name=nil,
    groups=["XXXXXX", "webservers", "XXXXXX", "default", "XXXXXX", "webservers", "XXXXXX", "default"],
    flavor_id="t1.micro",
    image_id="XXXXXX",
    kernel_id="aki-ace26f9c",
    key_name="artem",
    created_at=2012-01-30 10:44:32 UTC,
    monitoring=false,
    placement_group=nil,
    platform=nil,
    product_codes=[],
    private_dns_name=nil,
    private_ip_address=nil,
    public_ip_address=nil,
    ramdisk_id=nil,
    reason="User initiated (2012-01-30 12:01:57 GMT)",
    root_device_name=nil,
    root_device_type="ebs",
    security_group_ids=nil,
    state="stopped",
    state_reason={"code"=&gt;0},
    subnet_id=nil,
    tenancy="default",
    tags={"Name"=&gt;"grouptest"},
    user_data=nil
  &gt;
</pre>

The current parse code simply double counts groupSet by not distinguishing between the groupSet field in the reservation and the groupSet field in the item.

A simple fix is to ignore the groupSet field if the context is instancesSet.

With the applied patch duplicates are removed.

<pre>
Instance ID  Name        Zone        Public IP  Private IP  Flavor    Image         SSH Key  Sec Group              State  
i-XXXXXXXX   mastertest  us-west-2a                         t1.micro  ami-XXXXXXXX  artem    [webservers]           stopped
i-XXXXXXXX   grouptest   us-west-2a                         t1.micro  ami-XXXXXXXX  artem    [webservers, default]  stopped
</pre>