apache/spark — pull request #59 by CodingCat (2014-03-03 03:07:43+00:00)
Title: SPARK-1166: clean vpc_id if the group was just now created

Reported in https://spark-project.atlassian.net/browse/SPARK-1166

In some very weird situation (when new created group master_group and slave_group have valid vpc_id), user will receive the following error when running the spark-ec2 script

```

Setting up security groups...
ERROR:boto:400 Bad Request
ERROR:boto:
InvalidParameterValueInvalid value 'null' for protocol. VPC security group rules must specify protocols explicitly.fc56f0ba-915a-45b6-8555-05d4dd0f14ee
Traceback (most recent call last):
File "./spark_ec2.py", line 813, in
main()
File "./spark_ec2.py", line 806, in main
real_main()
File "./spark_ec2.py", line 689, in real_main
conn, opts, cluster_name)
File "./spark_ec2.py", line 244, in launch_cluster
slave_group.authorize(src_group=master_group)
File "/Users/nanzhu/code/spark/ec2/third_party/boto-2.4.1.zip/boto-2.4.1/boto/ec2/securitygroup.py", line 184, in authorize
File "/Users/nanzhu/code/spark/ec2/third_party/boto-2.4.1.zip/boto-2.4.1/boto/ec2/connection.py", line 2181, in authorize_security_group
File "/Users/nanzhu/code/spark/ec2/third_party/boto-2.4.1.zip/boto-2.4.1/boto/connection.py", line 944, in get_status
boto.exception.EC2ResponseError: EC2ResponseError: 400 Bad Request

InvalidParameterValueInvalid value 'null' for protocol. VPC security group rules must specify protocols explicitly.fc56f0ba-915a-45b6-8555-05d4dd0f14ee

```

The related code in boto is as following:

```
group_name = None
if not self.vpc_id:
group_name = self.name
group_id = None
if self.vpc_id:
group_id = self.id
src_group_name = None
src_group_owner_id = None
src_group_group_id = None
if src_group:
cidr_ip = None
src_group_owner_id = src_group.owner_id
if not self.vpc_id:
src_group_name = src_group.name
else:
if hasattr(src_group, 'group_id'):
src_group_group_id = src_group.group_id
else:
src_group_group_id = src_group.id
status = self.connection.authorize_security_group(group_name,
src_group_name,
src_group_owner_id,
ip_protocol,
from_port,
to_port,
cidr_ip,
group_id,
src_group_group_id)

```

So if we just create a new cluster, we should clean the vpc_id for the user