Repository: DataBiosphere/toil

Issue:
RealtimeLogging doesn't work with autoscaling
I'm a huge fan of the --realTimeLogging option as it's great for following progress and debugging.  It worked on Mesos with cgcloud, but does not seem to work at all on the AWS provisioner with autoscaling.  Is this fixable?  I believe @adamnovak is the original author...


Discussion:
It could just be a firewall issue. Can random AWS instances send packets to
your master? Or is it locked down by an AWS security group?

On Fri, Aug 11, 2017 at 9:05 AM, Glenn Hickey <notifications@github.com>
wrote:

> I'm a huge fan of the --realTimeLogging option as it's great for following
> progress and debugging. It worked on Mesos with cgcloud, but does not seem
> to work at all on the AWS provisioner with autoscaling. Is this fixable? I
> believe @adamnovak <https://github.com/adamnovak> is the original
> author...
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/BD2KGenomics/toil/issues/1818>, or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AE0_X77LI_Vs6A4DaUYNfEyF9agDYMj7ks5sXF-ggaJpZM4O0sx6>
> .
>

Keep in mind that there's no encryption or authentication on the real time
log messages. So someone on the Internet can inject things into your logs
if you open the port up globally.

On Sat, Aug 12, 2017 at 6:02 PM, Adam Novak <anovak@soe.ucsc.edu> wrote:

> It could just be a firewall issue. Can random AWS instances send packets
> to your master? Or is it locked down by an AWS security group?
>
> On Fri, Aug 11, 2017 at 9:05 AM, Glenn Hickey <notifications@github.com>
> wrote:
>
>> I'm a huge fan of the --realTimeLogging option as it's great for
>> following progress and debugging. It worked on Mesos with cgcloud, but does
>> not seem to work at all on the AWS provisioner with autoscaling. Is this
>> fixable? I believe @adamnovak <https://github.com/adamnovak> is the
>> original author...
>>
>> —
>> You are receiving this because you were mentioned.
>> Reply to this email directly, view it on GitHub
>> <https://github.com/BD2KGenomics/toil/issues/1818>, or mute the thread
>> <https://github.com/notifications/unsubscribe-auth/AE0_X77LI_Vs6A4DaUYNfEyF9agDYMj7ks5sXF-ggaJpZM4O0sx6>
>> .
>>
>
>


Resolving patch:
diff --git a/src/toil/provisioners/aws/awsProvisioner.py b/src/toil/provisioners/aws/awsProvisioner.py
--- a/src/toil/provisioners/aws/awsProvisioner.py
+++ b/src/toil/provisioners/aws/awsProvisioner.py
@@ -768,8 +768,12 @@ def groupNotFound(e):
                     web.authorize(ip_protocol='tcp', from_port=22, to_port=22, cidr_ip='0.0.0.0/0')
             for attempt in retry(predicate=groupNotFound, timeout=300):
                 with attempt:
-                    # the following authorizes all port access within the web security group
+                    # the following authorizes all TCP access within the web security group
                     web.authorize(ip_protocol='tcp', from_port=0, to_port=65535, src_group=web)
+            for attempt in retry(predicate=groupNotFound, timeout=300):
+                with attempt:
+                    # We also want to open up UDP, both for user code and for the RealtimeLogger
+                    web.authorize(ip_protocol='udp', from_port=0, to_port=65535, src_group=web)
         out = []
         for sg in ctx.ec2.get_all_security_groups():
             if sg.name == name and vpcId is None or sg.vpc_id == vpcId:


Test patch: