Package: mario
Have you ever wanted to use Python functions directly in your Unix shell? Mario can read and write csv, json, and yaml; traverse trees, and even do xpath queries. Plus, it supports async commands right out of the box. Build your own commands with a simple configuration file, and install plugins for even more!Mario is the plumbing snake 🐍🔧 helping you build data pipelines in your shell 🐢.FeaturesExecute Python code in your shell.Pass Python objects through multi-stage pipelines.Read and write csv, json, yaml, toml, xml.Run async functions natively.Define your own commands in a simple configuration file or by writing Python code.Install plugins to get more commands.Enjoy high test coverage, continuous integration, and nightly releases.InstallationMarioWindows support is hopefully coming soon. Linux and MacOS are supported now.Get Mario with pip:python3.7-mpipinstallmarioIf you’re not inside a virtualenv, you might get aPermissionsError. In that case, try using:python3.7-mpipinstall--usermarioor for more isolation, usepipx:pipxinstall--pythonpython3.7marioMario addonsThemario-addonspackage provides a number of useful commands not found in the base collection.Get Mario addons with pip:python3.7-mpipinstallmario-addonsIf you’re not inside a virtualenv, you might get aPermissionsError. In that case, try using:python3.7-mpipinstall--usermario-addonsor for more isolation, usepipx:pipxinstall--pythonpython3.7mariopipxinjectmariomario-addonsQuickstartBasicsInvoke withmarioat the command line.$marioeval1+12Given a csv like this:$cat<<EOF > hackers.csv
name,age
Alice,21
Bob,22
Carol,23
EOFUseread-csv-dictsto read each row into a dict:$marioread-csv-dicts<hackers.csv{'name':'Alice','age':'21'}{'name':'Bob','age':'22'}{'name':'Carol','age':'23'}Usemapto act on each input itemx:$marioread-csv-dictsmap'x["name"]'<hackers.csvAliceBobCarolChain Python functions together with!:$marioread-csv-dictsmap'x["name"] ! len'<hackers.csv535or by adding another command$marioread-csv-dictsmap'x["name"]'maplen<hackers.csv535Usexas a placeholder for the input at each stage:$marioread-csv-dictsmap'x["age"] ! int ! x*2'<hackers.csv424446Automatically import modules you need:$mariomap'collections.Counter ! dict'<<<mississippi{'m':1,'i':4,'s':4,'p':2}You don’t need to explicitly call the function withsome_function(x); just use the function’s name,some_function. For example, instead of$mariomap'len(x)'<<EOF
a
bb
EOF12try$mariomaplen<<EOF
a
bb
EOF12More commandsHere are a few commands. SeeCommand referencefor the complete set, and get even more frommario-addons.evalUseevalto evaluate a Python expression.%marioeval'datetime.datetime.utcnow()'2019-01-0101:23:45.562736mapUsemapto act on each input item.$mariomap'x * 2'<<EOF
a
bb
EOFaabbbbfilterUsefilterto evaluate a condition on each line of input and exclude false values.$mariofilter'len(x) > 1'<<EOF
a
bb
ccc
EOFbbcccapplyUseapplyto act on the sequence of items.$marioapply'len(x)'<<EOF
a
bb
EOF2chainUsechainto flatten a list of lists into a single list, likeitertools.chain.from_iterable.For example, after generating a several rows of items,$marioread-csv-tuples<<EOF
a,b,c
d,e,f
g,h,i
EOF('a','b','c')('d','e','f')('g','h','i')usechainto put each item on its own row:$marioread-csv-tupleschain<<EOF
a,b,c
d,e,f
g,h,i
EOFabcdefghiasync-mapMaking sequential requests is slow. These requests take 16 seconds to complete.%timemariomap'await asks.get ! x.json()["url"]'<<EOF
http://httpbin.org/delay/5
http://httpbin.org/delay/1
http://httpbin.org/delay/2
http://httpbin.org/delay/3
http://httpbin.org/delay/4
EOFhttps://httpbin.org/delay/5https://httpbin.org/delay/1https://httpbin.org/delay/2https://httpbin.org/delay/3https://httpbin.org/delay/40.51suser0.02ssystem16.460totalConcurrent requests can go much faster. The same requests now take only 6 seconds. Useasync-map, orasync-filter, orreducewithawait some_async_functionto get concurrency out of the box.%timemarioasync-map'await asks.get ! x.json()["url"]'<<EOF
http://httpbin.org/delay/5
http://httpbin.org/delay/1
http://httpbin.org/delay/2
http://httpbin.org/delay/3
http://httpbin.org/delay/4
EOFhttps://httpbin.org/delay/5https://httpbin.org/delay/1https://httpbin.org/delay/2https://httpbin.org/delay/3https://httpbin.org/delay/40.49suser0.03ssystem5.720totalConfigurationDefine new commands and set default options. SeeConfiguration referencefor details.PluginsAdd new commands likemapandreduceby installing Mario plugins. You can try them out without installing by adding them to any.pyfile in your~/.config/mario/modules/.Share popular commands by installing themario-addonspackage.Q & AWhat’s the status of this package?This package is experimental and is subject to change without notice.Check theissues pagefor open tickets.Why another package?A number of cool projects have pioneered in the Python-in-shell space. I wrote Mario because I didn’t know these existed at the time, but now Mario has a bunch of features the others don’t (user configuration, multi-stage pipelines, async, plugins, etc).https://github.com/Russell91/pythonpyhttp://gfxmonk.net/dist/doc/piep/https://spy.readthedocs.io/en/latest/intro.htmlhttps://github.com/ksamuel/Pypedhttps://github.com/ircflagship2/pypehttps://code.google.com/archive/p/pyp/