galaxyproject/pulsar — pull request #34 by erasche (2014-10-16 18:40:14+00:00)
Title: SSH Transport Implemented

Provides the file action "remote_rsync_transfer". This currently
*requires* you to use a `file_action_config` on the Galaxy side, as
extra configuration is required.

Galaxy job_conf.xml:

file_actions.yaml

Galaxy file_actions.yaml:

paths:
- path: /home/vagrant/
action: remote_rsync_transfer
ssh_user: vagrant
ssh_host: my.fqdn
ssh_port: 2222
ssh_key: |
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEAwWYGEOhiJgbWv8eBV2LJp0MCSgrAPeX9FbsGn1I+UC110PPW

This passes configuration information in every message to Pulsar. While
suboptimal in terms of message size (ssh key is ~3x the size of a normal
message), it opens the door to multiple galaxies connecting to a single pulsar,
still using rsync. Additionally it lets the Galaxy admin reconfigure the SSH
parameters without any interaction on the Pulsar admin's part (with the
exception of `ssh_key`, as of now). This will be very beneficial if the two
groups do not have a close working relationship.

## Future Work

- Per TODO in pulsar/managers/stateful.py it should be trivial to define a section of parameters that are automatically extracted from the XML/file_actions (e.g. under a key `meta`), which are handled much more generically than the current hardcoding of `ssh_key` everywhere.