Package: skypipe
Skypipe is a magical command line tool that lets you easily pipe data across terminal sessions, regardless of whether the sessions are on the same machine, across thousands of machines, or even behind a firewall. It gives you named pipes in the sky and lets you magically pipe dataanywhere.InstallingCurrently you need Python 2.6, ZeroMQ 2.0+ and the ability to compile extensions.
Then install with pip:$ pip install skypipeSetting upThe magic of skypipe requires a free dotcloud account. If you don’t have
one, you can easily create one for free. The first time you use skypipe,
you will be asked for credentials.Using SkypipeSkypipe combines named pipes and netcat and gives you even more power in a simpler tool. Here is a simple example using skypipe like you would a named pipe in order to gzip a file across shells:$ skypipe | gzip -9 -c > out.gzYour skypipe is ready to receive some data from another shell process:$ cat file | skypipeUnliked named pipes, however,this will work across any machines connected to the Internet. And you didn’t have to specify a host address or set up “listen mode” like you would with netcat. In fact, unlike netcat, which is point to point, you could use skypipe for log aggregation. Here we’ll used named skypipes. Run this on several hosts:$ tail -f /var/log/somefile | skypipe logsAnd then run this on a single machine:$ skypipe logs > /var/log/aggregateOr alternatively you can broadcast to multiple hosts. With the above, you can “listen in” by running this from your laptop, even while behind a NAT:$ skypipe logsAlso unlike netcat, you can temporarily store data “in the pipe” until you pull it out. In fact, you can keep several “files” in the pipe. On one machine load some files into it a named skypipe:$ cat file_a | skypipe files
$ cat file_b | skypipe filesNow from somewhere else pull them out, in order:$ skypipe files > new_file_a
$ skypipe files > new_file_bLastly, you can use skypipe like the channel primitive in Go for coordinating between shell scripts. As a simple example, here we use skypipe to wait for an event triggered by another script:#!/bin/bash
echo "I'm going to wait until triggered"
skypipe trigger
echo "I was triggered!"Triggering is just sending an EOF over the pipe, causing the listening skypipe to terminate. We can do this with a simple idiom:$ echo | skypipe triggerSoftware with a serviceThe trick to this private pipe in the sky is that when you first use skypipe, behind the scenes it will deploy a very simple messaging server to dotcloud. Skypipe will use your account to transparently find and use this server, no matter where you are.This all works without you ever thinking about it because this server is managed automatically and runs on dotcloud for free. It might as well not exist!This represents a new paradigm of creating tools that transparently leverage the cloud to create magical experiences. It’s not quite software as a service, it’s softwarewitha service. Nobody is using a shared, central service. The software deploys its own service on your behalf for you to use.Thanks to platforms like dotcloud (and Heroku), we can now build software leveraging features of software as a service that ispackaged and distributed like normal open source software.ContributingThere aren’t any tests yet, but it’s pretty well documented and the code
is written to be read. Fork and send pull requests. Check out the issues
to see how you can be most helpful.ContributorsJeff Lindsay <progrium@gmail.com>LicenseMIT