Package: trivio
![Trivio Logo](http://www.triv.io/images/trivio_logo.png)Example Workflow===We're making managing a big data project as simple as a "git push". Think of us as "Heroku for Big Data"With triv.io you can transform massive amounts of poorly structured data on a regular basis. Regardless of the size and whether your data exists in log files, multiple databases or even at the end of a web API we help you store, transform and share your data with others.Install triv.io and login---```$ pip install trivio$ trivio loginTriv.io uses github for authentication. You'll need a githubusername and password to proceed. Authentication is done directlywith the github servers, we never see or store your user name andpassword. Don't believe us? Have a look at the trivio client source.GitHub username:GitHub password:```Create your first project---```$ trivio create https://github.com/trivio/hello_worldCreating project hello_worldYou do not have write access to this projectwould you like to fork it?[Yn]: Yforking project to https://github.com/username/hello_worldcloning  https://github.com/username/hello_worldYour project is ready for use```Try it out---```~ $ cd hello_worldhello_world $ trivio simulate word_countsblah 1foo 120baz 767```Make some changes---Using your favorite editor let's make word_counts use a better tokenizer```hello_world $ lsword_counts.pyjhello_world $ vi word_counts.pyj```Change the file to look like this```@word_counts.mapdef map(doc, params,   tokenizer = re.compile('[^A-Z0-9_.]+', flags=re.I)):for word in tokenizer.split(doc['payload']):yield word, 1```Now run simulate again to verify the results```hello_world $ trivio simulate word_countsblah 1foo 120baz 767```Push your changes---Once you've tweaked your algorithm you need to push those changes to trivio```hello_world $ trivio push "Used a better tokenizer"changing to srobertson[master 37200ed] pushing1 files changed, 1 insertions(+), 1 deletions(-)Counting objects: 5, done.Delta compression using up to 8 threads.Compressing objects: 100% (3/3), done.Writing objects: 100% (3/3), 301 bytes, done.Total 3 (delta 2), reused 0 (delta 0)To git@github.com:srobertson/common_crawl_index.git4ebf177..37200ed  master -> masterSuccessfully pushed your changes to trivio.Your project is paused use 'trivio step' or 'trivio resume' to build usingyour changes.hello_world $```Compute a single segment---By default, all projects start off "paused", this prevents them from running up your compute bills until you're satisfied your code works the way you expect it to. You've already used the `trivio simulate` method which samples your data and returns you results. The next step is to build a segment ("a portion of your table"see triv.io data model for more details)```$ trivio step word_countsBuilding next segment 2009-9-17 00:00:00$```Notice that this command returns immediately, building a segment can be quite involved. You may be producing terabytes of data which you wouldn't want to download to your workstation.Monitoring Build Progress---```$ trivio watchTable                       Queued     Built Failed---------------------------------------------------.../common-crawl/parsed-output    0        1      0Next build: Never (project paused)```This will monitor your progress of your project. When the watch command spits out Next build Never. Hit `ctrl-c` to get back to your command line. As noted your project is paused so it won't build anything else until you either tell trivio to build a single segment with the step command or tell trivio to unpause your project. We'll get to unpausing in a momentQuerying tables---Now that trivio has had a chance to build part of your table we can query it for data. Tri.vio uses a sql like syntax for querying```$ trivio query "select * from word_counts"+----------+-------+|   key    | value |+----------+-------+| Zachary  | 22731 || Alfred   | 20477 || Gregory  | 17179 || Ned      | 16860 || Ulrich   | 15300 || Thomas   | 14995 |+----------+-------+```Put Triv.io on Autopilot---Once you're happy with your project, you can tell triv.io to keep your tables up to date automatically based on the schedules you defined in your scripts.```$ trivio resumeAuto building all tables, use 'trivio watch' to monitor progress```Where to go from here---That's the basic workflow* [Trivio data model](http://docs.triv.io/)* [Talking to new storages](http://docs.triv.io/)* [Interpreting mime types](http://docs.triv.io/)* Check out our [blog](http://docs.triv.io/) for cool projects, tips and tricks[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/trivio/trivio.client/trend.png)](https://bitdeli.com/free "Bitdeli Badge")