fog/fog — pull request #719 by aia (2012-01-31 02:09:29+00:00)
Title: Removing duplicates from reservation's groupSet

Fog::Compute::AWS::Server objects produced from describe_instances contain duplicates in the group's array.

As a result, tools built on Fog e.g knife-ec2 produce confusing output:

# knife ec2 server list
Instance ID Name Zone Public IP Private IP Flavor Image SSH Key Sec Group State
i-XXXXXXXX mastertest us-west-2a t1.micro ami-XXXXXXXX artem [webservers, webservers] stopped
i-XXXXXXXX grouptest us-west-2a t1.micro ami-XXXXXXXX artem [webservers, default, webservers, default] stopped

This patch fixes the describe_instances parser and eliminates duplicates.

Here are the details.

A sample AWS response to describe_instances request:

<?xml version="1.0" encoding="UTF-8"?>
<DescribeInstancesResponse xmlns="http://ec2.amazonaws.com/doc/2011-11-01/">
<requestId>51b74328-d2d7-4b02-867d-818fec883f7d</requestId>
<reservationSet>
<item>
<reservationId>XXXXXX</reservationId>
<ownerId>XXXXXX</ownerId>
--> <groupSet> <-- reservation groupSet
<item>
<groupId>XXXXXX</groupId>
<groupName>webservers</groupName>
</item>
</groupSet>
<instancesSet>
<item>
<instanceId>XXXXXX</instanceId>
<imageId>XXXXXX</imageId>
<instanceState>
<code>80</code>
<name>stopped</name>
</instanceState>
<privateDnsName/>
<dnsName/>
<reason>User initiated (2012-01-30 05:44:47 GMT)</reason>
<keyName>artem</keyName>
<amiLaunchIndex>0</amiLaunchIndex>
<productCodes/>
<instanceType>t1.micro</instanceType>
<launchTime>2012-01-30T05:00:34.000Z</launchTime>
<placement>
<availabilityZone>us-west-2a</availabilityZone>
<groupName/>
<tenancy>default</tenancy>
</placement>
<kernelId>aki-ace26f9c</kernelId>
<monitoring>
<state>disabled</state>
</monitoring>
--> <groupSet> <-- instance groupSet
<item>
<groupId>XXXXXX</groupId>
<groupName>webservers</groupName>
</item>
</groupSet>
<stateReason>
<code>Client.UserInitiatedShutdown</code>
<message>Client.UserInitiatedShutdown: User initiated shutdown</message>
</stateReason>
<architecture>x86_64</architecture>
<rootDeviceType>ebs</rootDeviceType>
<rootDeviceName>/dev/sda1</rootDeviceName>
<blockDeviceMapping>
<item>
<deviceName>/dev/sda1</deviceName>
<ebs>
<volumeId>XXXXXX</volumeId>
<status>attached</status>
<attachTime>2012-01-30T05:45:08.000Z</attachTime>
<deleteOnTermination>true</deleteOnTermination>
</ebs>
</item>
</blockDeviceMapping>
<virtualizationType>paravirtual</virtualizationType>
<clientToken/>
<tagSet>
<item>
<key>Name</key>
<value>mastertest</value>
</item>
</tagSet>
<hypervisor>xen</hypervisor>
</item>
</instancesSet>
</item>
<item>
<reservationId>XXXXXX</reservationId>
<ownerId>XXXXXX</ownerId>
--> <groupSet> <-- reservation groupSet
<item>
<groupId>XXXXXX</groupId>
<groupName>webservers</groupName>
</item>
<item>
<groupId>XXXXXX</groupId>
<groupName>default</groupName>
</item>
</groupSet>
<instancesSet>
<item>
<instanceId>XXXXXX</instanceId>
<imageId>XXXXXX</imageId>
<instanceState>
<code>80</code>
<name>stopped</name>
</instanceState>
<privateDnsName/>
<dnsName/>
<reason>User initiated (2012-01-30 12:01:57 GMT)</reason>
<keyName>artem</keyName>
<amiLaunchIndex>0</amiLaunchIndex>
<productCodes/>
<instanceType>t1.micro</instanceType>
<launchTime>2012-01-30T10:44:32.000Z</launchTime>
<placement>
<availabilityZone>us-west-2a</availabilityZone>
<groupName/>
<tenancy>default</tenancy>
</placement>
<kernelId>aki-ace26f9c</kernelId>
<monitoring>
<state>disabled</state>
</monitoring>
--> <groupSet> <-- instance groupSet
<item>
<groupId>XXXXXX</groupId>
<groupName>webservers</groupName>
</item>
<item>
<groupId>XXXXXX</groupId>
<groupName>default</groupName>
</item>
</groupSet>
<stateReason>
<code>Client.UserInitiatedShutdown</code>
<message>Client.UserInitiatedShutdown: User initiated shutdown</message>
</stateReason>
<architecture>x86_64</architecture>
<rootDeviceType>ebs</rootDeviceType>
<rootDeviceName>/dev/sda1</rootDeviceName>
<blockDeviceMapping>
<item>
<deviceName>/dev/sda1</deviceName>
<ebs>
<volumeId>XXXXXX</volumeId>
<status>attached</status>
<attachTime>2012-01-30T12:02:17.000Z</attachTime>
<deleteOnTermination>true</deleteOnTermination>
</ebs>
</item>
</blockDeviceMapping>
<virtualizationType>paravirtual</virtualizationType>
<clientToken/>
<tagSet>
<item>
<key>Name</key>
<value>grouptest</value>
</item>
</tagSet>
<hypervisor>xen</hypervisor>
</item>
</instancesSet>
</item>
</reservationSet>
</DescribeInstancesResponse>

The above request produces the following Fog::Compute::AWS::Server objects:

<Fog::Compute::AWS::Server
id="XXXXXX",
ami_launch_index=0,
availability_zone="us-west-2a",
block_device_mapping=[{"deviceName"=>"/dev/sda1", "volumeId"=>"XXXXXX", "status"=>"attached", "attachTime"=>2012-01-30 05:45:08 UTC, "deleteOnTermination"=>true}],
client_token=nil,
dns_name=nil,
groups=["XXXXXX", "webservers", "XXXXXX", "webservers"],
flavor_id="t1.micro",
image_id="XXXXXX",
kernel_id="aki-ace26f9c",
key_name="artem",
created_at=2012-01-30 05:00:34 UTC,
monitoring=false,
placement_group=nil,
platform=nil,
product_codes=[],
private_dns_name=nil,
private_ip_address=nil,
public_ip_address=nil,
ramdisk_id=nil,
reason="User initiated (2012-01-30 05:44:47 GMT)",
root_device_name=nil,
root_device_type="ebs",
security_group_ids=nil,
state="stopped",
state_reason={"code"=>0},
subnet_id=nil,
tenancy="default",
tags={"Name"=>"mastertest"},
user_data=nil
>,
<Fog::Compute::AWS::Server
id="XXXXXX",
ami_launch_index=0,
availability_zone="us-west-2a",
block_device_mapping=[{"deviceName"=>"/dev/sda1", "volumeId"=>"XXXXXX", "status"=>"attached", "attachTime"=>2012-01-30 12:02:17 UTC, "deleteOnTermination"=>true}],
client_token=nil,
dns_name=nil,
groups=["XXXXXX", "webservers", "XXXXXX", "default", "XXXXXX", "webservers", "XXXXXX", "default"],
flavor_id="t1.micro",
image_id="XXXXXX",
kernel_id="aki-ace26f9c",
key_name="artem",
created_at=2012-01-30 10:44:32 UTC,
monitoring=false,
placement_group=nil,
platform=nil,
product_codes=[],
private_dns_name=nil,
private_ip_address=nil,
public_ip_address=nil,
ramdisk_id=nil,
reason="User initiated (2012-01-30 12:01:57 GMT)",
root_device_name=nil,
root_device_type="ebs",
security_group_ids=nil,
state="stopped",
state_reason={"code"=>0},
subnet_id=nil,
tenancy="default",
tags={"Name"=>"grouptest"},
user_data=nil
>

The current parse code simply double counts groupSet by not distinguishing between the groupSet field in the reservation and the groupSet field in the item.

A simple fix is to ignore the groupSet field if the context is instancesSet.

With the applied patch duplicates are removed.

Instance ID Name Zone Public IP Private IP Flavor Image SSH Key Sec Group State
i-XXXXXXXX mastertest us-west-2a t1.micro ami-XXXXXXXX artem [webservers] stopped
i-XXXXXXXX grouptest us-west-2a t1.micro ami-XXXXXXXX artem [webservers, default] stopped