Package: backstage
By © Jorge Royan /http://www.royan.com.ar,CC BY-SA 3.0,LinkPyrustic BackstageThree-speed scripting language and task automation toolThis project is part of thePyrustic Open Ecosystem.InstallationDemoLatestModulesTable of contentsOverviewStructure of the script fileSpawning processes and branching subtasksData types and control flowNamespaces and persistenceVariable interpolation and escapingEnvironment variables and language syntaxFile and directory manipulationException handling and testsInterfacing with PythonException handling and testsCommand line interface and developer experienceMiscellaneousDemoInstallationOverviewBackstageis a cross-platformautomation toolthat looks for abackstage.tasksfile in the current working directory to run a specific task defined in that file on demand. A task can be a sequence or pipeline of processes to be spawned, instructions for performing file and directory manipulation, or something more sophisticated.Thebackstage.tasksfile uses theJesth(JustExtractSectionsThenHack) file format that acts like a brokenINI fileparser that only extract sections each made of a header and a body which is just a list of lines.Using an eponymousthree-speed scripting languagedesigned for the automation tool, the programmer can, inside thebackstage.tasksfile, define, coordinate and use the various resources at his disposal to automate things.The three-speed scripting language concept is inspired from the three forward gear ratios of early automobilestransmission system. In the following subsections, we will explore each metaphorical gear of theBackstagescripting language, then we will briefly expose the automation tool itself.First gearIn first gear, abackstage.tasksfile is intended to store a list of tasks related to a specific project, each task exposing a list of commands or subtasks to be executed. Here, a command represents a process or pipeline of processes to be spawned.Environment variablescan be used in commands viavariable interpolation. No other logic is involved.Example[task1]
# three commands to run sequentially
$ git commit -m 'Update'
$ python -m my.package.module
$ program1 arg1 | program2 {HOME}
---
# run the subtask 'task2'
& task2
---
# run the subtask 'task3' in a new thread
~ task3

[task2]
$ program val1 {CWD}
$ git push origin master

[task3]
# some heavy computation
$ engine -x 5000
$ engine --cleanup

[_task4]
# This is a private task (with an underscore as prefix)
$ clean dirSecond gearIn second gear, thebackstage.tasksfile not only stores the tasks like in first gear, but here logic intervenes, variables are defined, control flow is used, built-in commands are called, et cetera. Basically, in second gear,Backstageunleashes its power and allows the programmer to anticipate problems, make sophisticated combinations of subtasks, in short to write a realscript to automate things.Example[task1]
# commit changes
$ git commit -m 'Update'
---
# tell user if 'Commit' has been success
if R == 0
    # print 'Success !'
    : Success !
else
    : Failed to commit changes
---
# say hello ten times
set age = 42
set name = `John Doe`
from 1 to 10
    $ python -m say.hello {name} {age}
---
# create a file in user home
set pathname = {HOME}/iliad.txt
create file pathname
---
# append some data to a file
set data = `\nHello World !`
append data to pathname
---
# browse current working directory
browse files and dirs in CWD
    : Directory -> {R}
    : Files ->
    for item in files
        : - {item}
    : Dirs ->
    for item in dirs
        : - {item}Third gearIn third gear, in addition with whatever can be done with previous gears, the programmer can directly from thebackstage.tasksfile, callPythonfunctions with arguments and get the return ! Thanks to this third gear, any too complex or overly verbose calculation can be written inPythonand called fromBackstage. Thisfunctionalityalone proves thatBackstageis all about making the programmer's life better, not pretending to replace existing mature solutions that actually work.Example[task1]
interface with package.coffeemaker alias coffeebro

set sugar_cubes (int) = 1 + 1 + 1
set extra = `milk`

call coffeebro.make(sugar_cubes, extra, 42)

if R == 1
    : Coffee successfully made !
else
    : Oops, failed to make coffee...
    : Exception -> {EXCEPTION}
    : Traceback -> {TRACEBACK}Automation toolThe scripting language help to define tasks in thebackstage.tasksfile that is intended to be consumed by the automation tool. As an automation tool,Backstageexposes acommand line interfacethat allows the user todiscoveravailable tasks,runa task with arguments, read a taskdocumentation, use aglob-like syntax tosearchfor a task by its name or by a keyword (case-insensitive mode) that is part of the documentation of the task, et cetera.Example 1Let's assume that there is abackstage.tasksfile in the directory/home/alex/project. Thisbackstage.tasksfile contains three public tasks and one private tasks (prefixed with an underscore).This example shows how one could use the automation tool to run a task defined in thebackstage.tasksfile:$cd/home/alex/project

$backstage-c
Availabletasks(3):make_coffeetask1task2

$backstagemake_coffee
Makingcoffee...

$backstagemak*
Makingcoffee...

$backstagemake_coffeesugar=3Makingcoffeewith3sugarcubes...Example 2This is the contents of abackstage.tasksfile located at/home/alex/project:[task]
# define 'x' as a variable with an 'int' assignment tag
set x (int) = 1 + 1 + 1

# default name (by default, the assignment tag is 'str')
set default_name = `John Doe`

# print some text
: Hi and Welcome !
: I can print your name {x} times in a row !

# take user input (always a 'str')
> name : `What is your name ? `

# control flow
if name == EMPTY
    set name = {default_name}

# iteration
from 1 to x
    : {R} - Ave {name} !

# branch subtask '_task2' and pass it an argument
& _task2 {name}


[_task2]
# define the variable 'name' (first argument passed to this task)
set name = {ARGS[0]}

# Just say Goodbye !
: Goodbye {name} !Let's run the task namedtaskfrom the command line:$cd/home/alex/project

$backstagetask
HiandWelcome!
Icanprintyourname3timesinarow!
Whatisyourname?Alex1-AveAlex!2-AveAlex!3-AveAlex!
GoodbyeAlex!Note:You can reproduce this example as it. JustinstallBackstage, copy-paste the script in abackstage.tasksfile, then runbackstage taskin the command line.And moreThere is more to talk aboutBackstage, like the ability to embed documentation and tests in thebackstage.tasksfile and access them from the command line. Backstage is enough versatile to do the job of a trivial task runner or to automate things with its scripting language that has a built-in bridge to the powerful Python ecosystem.In the following sections, we will explore this project in depth. You can also jump to thedemoto start playing withBackstage!Structure of the script fileAs stated in theOverviewsection, abackstage.tasksfile is basically aJesthFile.In abackstage.tasksfile, a section represents a task. The section title is the name of the task and the section body is made of commands to run and the constructs of theBackstagescripting language. A valid task name is an alphanumeric string that can contains an underscore.You can write here at the top of the script,
a description of the script,
the date of its creation,
or any useful information.

[task1]
# body of task1
...

[task2]
# body of task2
...

[_private]
# prefix a task name with an underscore
# to turn it into a private task that
# won't appear in the list of available tasks
# when you will type 'backstage --check' in the command lineAs you can guess, a line starting with#is a comment. But this is only true inside a task body, because in fact not all sections are tasks as described before: a section can also be an embedded test or documentation.Embedded documentationYou can embed documentation inside thebackstage.tasksfile. To create a documentation for a task, create a section which name is postfixed with.help:[task1]
pass


[task1.help]
This is the description line.

Usage:
    backstage task <option> <path>

Options:
    -m, --msg       Show message
    -x, --exit      Exit blah blahFrom thecommand line, you can read the documentation of an arbitrary task:$backstage-htask1
Thisisthedescriptionline.

Usage:backstagetask<option><path>

Options:-m,--msgShowmessage-x,--exitExitblahblahEmbedded testsYou can embed tests inside thebackstage.tasksfile. To create a test for a task, create a section which name is postfixed with.test:[task1.test]
# perform some test here

# ...

assert some_var == some_varFrom thecommand line, you can run the test of an arbitrary task:$backstage--testtask1or the tests of a bunch of tasks:$backstage--testtask1task2task3or run all tests defined in thebackstage.tasksfile:$backstage--testSpawning processes and branching subtasksYou can write commands to spawn a process or a pipeline of processes:[task]
# spawn Git to perform a 'commit'
$ git commit -m "Update"

# spawn a pipeline of three processes
$ program1 arg1 | program2 arg2 | program3Commands to spawn processes support variable interpolation:[task]
# use HOME environment variable
$ ls {HOME}

# access the first index of the ARGS list
$ program {ARGS[0]}You canpushdata to theinputof a process:[task]
# define variables name and age
set name = `John Doe`
set age (int) = 40 + 2

# name and age will be pushed to the input of the next spawned process
push name age
$ program1 {HOME}

# from now you can access via the environment variable R,
# the return code (exit status code)
: The return code is {R}You cancapturea process:[task]
# you can capture a process,
# so you will get a direct access to the output and error
($) program2

# print the content of OUTPUT and ERROR
: Output -> {OUTPUT}
: Error -> {ERROR}You can redirectSTDOUTandSTDERR:[task]

# redirect STDOUT and STDERR
set STDOUT = `/path/to/file_out`
set STDERR = `/path/to/file_err`

$ program1

---

# cross platform DEVNULL:
set STDOUT = /dev/null

$ program2Branching subtasksYou can branch a subtask and pass arguments to it:[task1]
# branch 'task2'
& task2 {HOME}

[task2]
# from task2, we can access arguments passed to it.
# ARGS (environment variable) is a list of arguments.
: Arguments -> {ARGS}Whenever you branch a subtask, a new instance of this subtask is created, with its own variables. You can share data with a subtask with one of these three ways:pass arguments to the subtask while branching it;use the globalnamespace;use the databasenamespace.A subtask can also return data that is cached in theRenvironment variable:[task1]
& task2 "John Doe"
# from now, R contains 'Hello John Doe !'

[task2]
set result = `Hello {ARGS} !`
return resultMultithreadingBranching a subtask is done in the main thread. But one can create a new thread for a subtask:[task1]
# run an instance of task2 in a new thread
~ task2

# the next command won't wait 'task2' to complete
$ git commit -m "Update"


[task2]
# this task sleeps for 5 seconds
sleep 5Data types and control flowIn the next subsections we will talk about data types then control flow.Data typesBackstagesupportsvariablesand let the programmer set, use, clear, and drop variables.Under the hood,Backstageworks with fivePythondata types:str,list(one-dimensional),dict(one-dimensional),int, andfloat. But these data types aren't intended to be directly used by the programmer. Instead,assignment tagsare used to tell the interpreter how to treat a variable.Theseassignment tagsare:raw,str,list,dict,int,float,date,time,dtime, andtstamp.[task]
# this is a string
set var = 42

# this is another string
set var (str) = `Hello World`

# this is a list
set var (list) = `reg green blue`

# this is the same list but edited
set var[0] = `yellow`
# var -> yellow green blue

# this is a dict
set var (dict) = `user="John Doe" age=42 location=Kernel`

# this is the same dict but edited
set var.user = `Jane Doe`
# var -> user='Jane Doe' age=42 location=Kernel

# this is an integer
set var (int) = 40 + 2
# var -> 42

# this is a raw string
set regex (raw) = `[\S\s]*?`

# get the current timestamp
set now (int) = {NOW}
# now -> 1662569326

# convert it into datetime
set var (dtime) = {now}
# var -> 2022-09-07 17:48:46

# go from a datetime to timestamp
set var (tstamp) = `2022-09-07 17:48:46`
# var -> 1662569326

# extract the time part of a timestamp
set var (time) = 1662569326
# var -> 17:48:46Note that all variables have astringrepresentation andbackticksare used optionally as delimiters that will be ignored by the interpreter. So you can put backticks around an integer, and you can insert a list in a string.Here documentBackstagesupportshere documentfor strings inside thebackstage.tasksfile:[task]
# this is a text with two lines
set text (str) = `First line\nSecond line`

# this is another text with three lines
set text = `January\nFebruary\nMarch`

# this is not a text with two lines
set var (raw) = `Hello\nWorld`

# this is not a text with two lines
set var (str) = `Hello\\nWorld`Variable interpolationVariable interpolationis supported with the ability to access from a list or a string, the value at an arbitrary index, or from a dictionary, the value of a key.[task]
# let's play with 'str' variables
set x = `red`
set var = `{x} green blue`
# var -> red green blue

# get the value of element at index 0
set value = {var[0]}
# value -> r

---

# let's play with a list
set x = `red`
set var (list) = `{x} green blue`

# get the value of element at index 0
set value = {var[0]}
# value -> red

# get the value of elements from index 1 to the end
set value = {var[1:]}
# value -> green blue

---

# let's play with a dict
set x (int) = 40 + 2
set var (dict) = `name="John Doe" age={x}`

# get the value of key 'name'
set value = {var.name}
# value -> John Doe

# get the value of key 'age'
set value = {var.age}
# value -> 42

---

# cancel the variable interpolation
set var1 = `Hello` 
set var2 = `{{var1}} World`
# var2 -> {var1} WorldControl flowBackstageimplements conditionals and loops. A wide range of operators are available to compare values.Conditionals[task]
set var1 = 1
set var2 = 1
set x = 2
set regex (raw) = `[\S\s]*?`
set text = `Hello world`
set y = `Hello`

# conditionals support the classic
# operators: == != > < >= <= 
if var1 == 1
    $ program1
elif var2 == x
    $ program2
else
    $ program3
   
# Backstage supports logical and/or 
if var1 == var2 and var1 >= 3
    $ program1

# use the 'matches' operator
# or the negated one: !matches
if regex matches text
    : Matched !
elif regex !matches text
    : Mismatched !
    
# you can use the in operator
# and also the negated one: !in
if y in text
    pass
elif y !in text
    passLoops[task]

# From To loop
from 10 to 0
    : {R}
    
# For loop - iterate over a string
set text = `Hello World`
for char in text
    # N is an environment variable that serves as counter
    # for all loops
    : {N}- {char}

# For loop - iterate over a list
set data (list) = `red green blue`
for item in data
    : {item}
    
# For loop - iterate over a dict
set data (dict) = `name="John Doe" age=42`
for item in data
    : key -> {item[0]}  value -> {item[1]}
    
# For loop - iterate over a file
set path = `/home/alex/iliad.txt`
for line in path (file)
    : Line {N}
    : {line}
    :
    
# while loop
set var = 1
while var == 1
    : One Time Hello
    break
    
# browse loop
set path = `/home/alex`
browse files and dirs in path
    : Directory -> {R}
    for item in files
        : {item}
    for item in dirs
        : {item}Namespaces and persistenceNamespacesare implemented inBackstageto provide convenient management of variables by defining threescopes:LforLocalscope;GforGlobalscope;DforDatabasescope.By default, variables exist in theLocalnamespace and are only accessible to the running task.To share data with a subtask, one can expose arbitrary variables that will be copied into theGlobalnamespace which is readable and writable (thread-safe) by all subtasks.[task]
# by default, variables are defined in Local,
# i.e. they are only visible in the scope of the current task
set var = 42
: Var contains {var}
: Var still contains {L:var}

# you can make local variables public
expose var
# from now, you can get a thread-safe access to var 
# from any running task:
: Global var contains {G:var}

# branch _task2
& _task2

[_task2]
: I got {G:var} !Data can also bepersisted:[task]
set var = 42
store var

# from now, 'var' can be accessed by all tasks
# in this runtime but also in future runtimes
: Var contains {D:var}

# if you aren't sure about the existence
# of a variable, just do this:
default var

# it also works with a bunch of variables:
default var1 L:var2 D:var3 G:var4

# from now, if 'var1' hasn't been manually defined
# by the programmer, it will be
# automatically initialized and
# its value will be an empty stringPersisted variables are stored in.backstage/database.json.Variable interpolation and escapingDuring the string interpolation of a command that spawn processes or branch a subtask, variables that are of thestrtype are automaticallyshell-escaped.[task]
# This is a 'str' variable (backticks aren't quotes, by the way!)
set name = `John Doe` 
set colors (list) = `red green blue`

: Welcome {name} !
# Welcome John Doe

$ program name={name} -c {colors}
# program name='John Doe' -c red green blue

# Notice the quotes automatically added around the name John DoeEnvironment variables and language syntaxEnvironment variables are local to each instance of task. They are defined as uppercase strings. One can edit their contents but can't create new environment variables.This is the exhaustive list of environment variables:VariablesDescriptionARGSList of arguments passed to this task from the command lineCWDCurrent working directoryDATEThe current date in theYYYY-MM-DDformatEMPTYJust an empty stringERRORError string from a process previously spawnedEXCEPTIONName of the last exception raisedFALSEThe integer0HOMEThe path to$HOME. Example:/home/alexLINEThe current line (1-based numbering) of execution in the task bodyNCounter forwhile,for,from, andbrowseloopsNOWCurrent timestamp in secondsONEThe integer1OSThe running operating system:aix,linux,win32,cygwin,darwinOUTPUTOutput string from a process previously spawnedRThereturnof Python functions, built-in commands, statements, constructs, or process exit status codesRANDOMRandom integer between0and255(closed interval)SPACEOne spacecharacterSTDERRUse this variable to performSTDERRredirectionSTDINUse this variable to performSTDINredirectionSTDOUTUse this variable to performSTDOUTredirectionTASKThe name of the currently running taskTIMEThe current time in theHH:MM:SSformatTIMEOUTTimeout in seconds for commands that spawn processes. Default value:30secondsTMPTemporary directory.Attention, this directory will automatically disappear at the end of the runtime ! So think twice before moving files insideTRACEBACKTracebackof the last exception raisedTRASHPath to the trash:$HOME/PyrusticData/trashTRUEThe integer1ZEROThe integer0Note that theTRACEBACKandEXCEPTIONvariables are cleared after the next successful command.Backstagealso generates for convenience,ARG0,ARG1,ARGx, according to the contents ofARGS. For example, ifARGScontains two arguments, you can expect thatARG0andARG1exist.Language syntaxIn this section we will explore the built-in commands, statements, keywords, symbols, and language constructs that makeBackstage.Note that wherever a built-in command or statement expects avariablethat will beread, for convenience you can instead of supplying a variable name, define aninlineintorfloatliteral.Also, consider that theRenvironment variable is your friend, since it is used to cache the data returned by a statement, a construct, or a command.APPENDAppend data to a file.Usage:append <var> to <filename_var>ASSERTTest is a condition is true.Usage:assert (<var1>|<regex_var>) (==|!=|<=|>=|<|>|in|!in|rin|!rin|matches|!matches) <var2> [and|or] ...Example:assert regex_var matches text_var and var1 in listNote that!is used to express negation andrinis a Regex-basedin. A regexly-in operator ;)BRANCHBranch a subtask. The syntax is similar to the one to spawn processes, i.e., a string of words. The syntax supports variable interpolation.Usage:& <subtask> [<argument> ...]Example:& subtask1 name="John Doe" age=42 city={city}In this example, thecityvariable will be automatically shell-escaped during its interpolation.BREAKBreak a loop.Usage:breakBROWSELoop construct to browse a directory.Usage:browse [files] [and] [dirs] in <dirname_var>Example:[task]
browse files and dirs in dirname
    : Root -> {R}
    for item in files
        : {item}
    for item in dirs
        : {item}
        
browse files in dirname
    pass
    
browse dirs in dirname
    passCALLCall aPythonfunction fromBackstagewith arguments, then get the return !Usage:call <module>.<function>[(<argument_var>, ...)]Example:[task]
# interface with the Python module
interface with package.coffee_module alias coffeemaker

# call the 'make' function with arguments then get the return
call coffeemaker.make(sugar_cube, extra, 42)
: Result -> {R}CDChange directory.Usage:cd <dirname_var>CHECKReturn the data type (str,list,dict,int,float) of a variable if it exists, else return an empty string.Usage:check <var>Example:[task]
check myvar
if R == EMPTY
    : This variable doesn't exist at all !
else
    : 'myvar' exists, its data type is {R}CLEARClear the content of a variable or a list of variables.Usage:clear <var> ...Example:clear var1 var2 var3COMMENTComment.Usage:# <comment>CONFIGRead and write configuration options (FailFast,ReportException,ShowTraceback,TestMode,AutoLineBreak).Usage:config <option> ...Example:[task]
config FailFast=1 AutoLineBreak=0
config TestMode
if R == 1
    : Test Mode On
elif R == 0
    : Test Mode OffCOPYCopy a file or a directory tree to a new destination.Usage:copy <src_path_var> to <dest_path_var>COUNTCountchars,items, andlinesin the content of a variable or inside a file (if the(file)tag is applied).Usage:count (chars|items|lines) in (<var>|<filename_var>) [(file)]Example:[task]
set path = /home/alex/iliad.txt
count chars in path (file)
if R == 0
    : The file is empty !CREATECreate a new file or directory.Usage:create (dir|file) <path_var>DEFAULTDefine an empty variable (or a bunch of variables) if it doesn't exist yet in the namespace.Usage:default <var> ...Example:[task]
# default two variables in the Local namespace
default var1 L:var2

# default one variable in the Database namespace
default D:name

# from now, 'D:name' can be safely accessed
# even though the 'database.json' file supposed
# to contain the 'name' value was inadvertently deleted.DROPDestroy a variable (or a bunch of variables).Usage:drop <var> ...ELIFPart of theifconditional construct.Usage:elif (<var1>|<regex_var>) (==|!=|<=|>=|<|>|in|!in|rin|!rin|matches|!matches) <var2> [and|or] ...ELSEPart of theifconditional construct.Usage:elseENTERInvite user to submit data.Usage:> [<var> [: <text>]]Example:[task]

> name : Please enter your name 
# have you spotted the space at the end the line above ?

# the same line can be rewritten like this:
> name : `Please enter your name `
# backticks serve as delimiters that will be ignored

# this one is also possible:
set msg = `Please enter your name `
> name : {msg}

# even this:
set info (dict) = name="John Doe" age=42
> info.name : `Please enter your name`EXITExit.Usage:exitEXPOSECopy a variable (or a bunch of variables) into theGlobalnamespace.Usage:expose <var> ...FAILDeliberately fail. It breaks the running task and mark it as a failure.Usage:failFINDFind files and or directories paths.Usage 1:find [all] (paths|files|dirs) in <dirname_var>Usage 2:find ... matching <regex_var>Usage 3:find ... [and] (accessed|modified|created) (at|after|before|between) <timestamp_var> [and <timestamp_var>]Example:find files in dirname matching regex and accessed between timestamp1 and timestamp2FORAforloop to iterate the content of a variable or the content of a file (if you apply the(file)tag).Usage:for (char|item|line) in (<var>|<filename_var>) [(file)]Example:[task]
# this code iterates over each character of the Iliad,
# and outputs it as it,
# with one twist: each line starts with its index (0-based)

set path = `/home/alex/iliad.txt`

# the print statement (:) won't anymore
# automatically add a line break !
config AutoLineBreak=0

for line in path (file)
    : `{N} `
    for char in line
        : {char}
    : \nFROMA loop to go from an integer to another one. If thestartinteger is superior to theendinteger, the count will decrease.Usage:from <start> to <end>Example:[task]
from 10 to 0
    # here, N will go from 0 to 10
    # but R will go from 10 to 0
    # because N is a counter for all loops
    # while R is a cache for whatever is returned
    # by a command, a statement, or a construct
    : {N}\t{R}

# As you can see, I can add a Tab \t since
# Backstage supports natively here document ;)

# To get a simple backslash followed by a 't':  \\tGETGet achar, anitem, or alineat indexx(including negative index) from a target. The target can be the content of a variable or a file (if you apply the(file)tag).Usage:get (char|item|line) <index> from (<var>|<filename_var>) [(file)]IFConditional construct.Usage:if (<var1>|<regex_var>) (==|!=|<=|>=|<|>|in|!in|rin|!rin|matches|!matches) <var2> [and|or] ...Example:[task]
default var1 var2 var3 var4
if var1 == var2 or var3 == var4
    pass
elif EMPTY == EMPTY
    pass
else
    passINTERFACEInterface with aPythonmodule.Usage:interface with [<package>.]<module> [alias <name>]Example:[task]
interface with package.mymodule alias module
default var1 var2
call module.function(var1, var2)LINEDraw a line.Usage:(=|-) ...Example:----------or==========MOVEMove a file or a directory tree to a new destination.Usage:move <src_path_var> to <dest_path_var>PASSPlaceholder for the code that you might write in the future. This statement does nothing. It is the same as the eponymous one inPython.Usage:passPOKEPoke a file or directory to get access to adictof properties if this path exists. Available properties:sizemtimectimeatimenlinkuidgidmodeinodev.Usage:poke <path_var>Example:[task]
set path = /home/alex/iliad.txt
poke path
if R == EMPTY
    : Oops ! This file doesn't exist
else
    : File size -> {R.size}PREPENDPrepend data to a fileUsage:prepend <var> to <filename_var>PRINTPrint data. You can use backquotes as delimiters. This statement supports variable interpolation.Usage:: <text>Example:[task]
:  Hello World ! 
# Have you spotted the two extra spaces characters ?

: ` Hello World ! `
# hehehe, got you ! ;)PUSHPush variables into the input of the next spawned process.Usage:push <var> ...READRead all or a specific line index (including negative index) from a file.Usage:read (*|<index>) from <filename_var>REPLACEReplace some pattern in a text with a replacement value.Usage:replace <regex_var> in <text_var> with <replacement_var>RETURNReturn from a task with a value.Usage:return [<var>]SETDefine a new variable or update the content of an existing variable. You don't can't specify a data type, but instead you can apply an assigment tag that is one of:(raw)(str)(list)(dict)(int)(float)(date)(time)(dtime)(tstamp). Note that backquotes can be used as delimiters for the value (right side of the equal sign). These delimiters will be ignored. Backticks aren't quotes. This statement supports variable interpolation.Usage:set <var> [(raw)|(str)|(list)|(dict)|(int)|(float)|(date)|(time)|(dtime)|(tstamp))] = <value>Example:set var (int) = 1 + 2SLEEPSleep forxseconds.Usage:sleep <seconds>SPAWNSpawn a new process.Usage:$ <program> [<argument> ...]Example:$ program1 arg {var} | program2SPLITSplit with a regex pattern a text into a list.Usage:split <text_var> with <regex_var>SPOTCount the number of occurrences of a regex pattern inside a text.Usage:spot <regex_var> in <text_var>STOREStore a variable (or a bunch of variables) in theDatabasenamespace. A stored variable can be accessed like this:D:varUsage:store <var> ...THREADBranch a subtask... but in a new thread.Usage:~ <subtask> [<argument> ...]WHILEThewhileloop. Usebreakto break it, and checkNif you need a counter.Usage:while (<var1>|<regex_var>) (==|!=|<=|>=|<|>|in|!in|rin|!rin|matches|!matches) <var2> [and|or] ...WRITEErase the content of a file to write some data inside.Usage:write <var> to <filename_var>File and directory manipulationLet's explore how file and directory manipulatin is performed withBackstage.Resource creationCreate a file:[task]
# create a file
set path = /home/alex/iliad.txt
create file pathCreate a directory:[task]
# create a directory
set path = /home/alex/new/directory
create dir pathFile edition[task]
set path = /home/alex/iliad.txt
set var = Hello World

# write data
write var to path

# append data to a file
append var to path

# prepend data to a file
prepend var to pathRead the content of a file[task]
set path = /home/alex/iliad.txt

# read all from 'iliad.txt'
read * from path
: {R}

# read the line at index 3
set index (int) = 1 + 1 + 1
read index from path
: {R}

# just want to read the last line ?
read -1 from path 
: {R}Iterating the content of a file[task]
set path = /home/alex/iliad.txt

# iterate over the characters in a file
for char in path (file)
    : Character -> char

# iterate over the lines in a file
for line in path (file)
    : {line}Browse a folder[task]
set folder = /home/alex

browse files and dirs in folder
    : Directory -> {R}
    for item in files
        : {item}
    for item in dirs
        : {item}Find resourcesThefindstatement is like Glob but on steroid:[task]
set folder = /home/alex
set regex (raw) = `[\S\s]*?`
set timestamp1 = 1223322233

find files in folder matching regex and accessed between timestamp1 and NOW
: Results -> {R}Read resource propertiesYou can get fromBackstagethe properties of an arbitrary resource, like its size:[task]
set path = /home/alex/iliad.txt

# poke a file
poke path
: Creation timestamp -> {R.ctime}
: Size -> {R.size}Interfacing with PythonInterfacing withPythonis as simple as this:[task]
interface with python.module as my_module
set name = `John Doe`
set age (int) = 40 + 2
call my_module.function(name, age)
: Return -> {R}Allowed return data types:str,list(one-dimensional),tuple(one-dimensional),dict(one-dimensional),int, andfloat.Pythonfunctions can also returnTrue,False, andNone, which will be converted to1,0and anempty string, respectively.Exception handling and testsWhenever an exception is raised, the variablesEXCEPTIONandTRACEBACKare updated andBackstagecontinues calmly its execution.Note that the variablesTRACEBACKandEXCEPTIONare cleared after the next successful command.If you want the execution to stop whenever an exception is raised, just set1to theFailFastconfiguration option.[task]
config FailFast=1If you want to read a report of an exception when it's raised, just set1to theReportExceptionconfiguration option.[task]
config ReportException=1If you want to read the verbosetracebackof an exception when it's raised, just set1to theShowTracebackconfiguration option.[task]
config ShowTraceback=1You can edit these configuration options in the same command:[task]
config FailFast=1 ReportException=1 ShowTraceback=0You can read the current value of an arbitrary configuration option:[task]
config FailFast
: FailFast -> {R}Debug modeInstead of manually setting theReportExceptionconfiguration option to1, you can simply run a task in debug mode:$backstage-dtaskargTestsTo create a test, just postfix.testto the name of a task. Then from the command line, just run the testbackstage --test task.Example[task]
set val (int) = {ARGS[0]} + {ARGS[1]}
return val

[task.test]
# here we branch the task with the arguments 40 and 2
& task 40 2
# we expect 42 as return 
assert R == 42Command line interface and developer experience$backstage--help
WelcometoPyrusticBackstage!
Ultimatetaskautomationtoolforhackers.

Usage:backstagebackstage<task>[<arg>...]backstage<option>[<arg>...]Options:-i,--introShowfileintroductorytext-c,--checkShowthelistoftasks-C,--CheckShowthedescriptivelistoftasks-d,--debug<task>[<arg>...]Runtaskindebugmode-t,--test[<task>...]Runtests-T,--Test[<task>...]Runtestsindebugmode-s,--search<task>Searchforataskbyitsname-S,--Search<task>Searchforataskbykeyword-h,--help[<task>]ShowhelptextThe<task>stringcanuseaglob-likesyntaxthatallowswildcards'*'and'?'.Therefore,'task1'isidenticalto'task*'.Visitthewebpage:https://github.com/pyrustic/backstageDeveloper experienceBackstagewill do its best to help you understand raised exceptions:$backstagetask1
ZeroDivisionErroratline3of[task1]!
divisionbyzero

$backstagetask2
InterpretationErroratline7of[task2]!
Usage:sleep<seconds>When you runBackstagein the loop mode, you can enjoy the autocomplete functionality (use the Tab key to complete your input) and also the history functionality (use Up and Down arrows).$backstage
WelcometoPyrusticBackstage!
Ultimatetaskautomationtoolforhackers.
Press'Ctrl-c'or'Ctrl-d'toquit.
Type'--help'or'-h'toshowmoreinformation.(backstage)task(TabTab)tasktask1task2task3(backstage)--h(Tab)...MiscellaneousIn the following subsections, we will explore some miscellaneous information.DogfoodingBackstageitself as a project relies on abackstage.tasksfile (check the root of this repository). You are reading a document aboutBackstagethat has been updated withBackstage!DependenciesBackstagerelies on thesePythonpackages:Subrunto spawn new processes;Sharedto store data;Jesthto parsebackstage.tasksfiles;Oscanto extract tokens from the script.IndentationFour (4) spaces byindent. Period.Python 3Inside the script file, you don't have to typepython3in a command to spawn thePythoninterpreter. Just typepythonto spawn the same interpreter that is runningBackstage:[task]
$ python -m my.package.moduleShellBackstagedoesn't rely on anyShell. But you can still pipe commands !Example, if you want to change the current working directory:[task]
# instead of doing this
$ cd {HOME}

# use the built-in 'cd'
cd HOME

# you can still spawn programs commonly used in the shell
$ ls

# or make this complex stuff (successfully performed on Ubuntu)
$ python -m this | tail --lines=+3 | sortData cacheBackstagestores data in an automatically created directory.backstagelocated in the current working directory. Inside this directory you can find theexecution.loganddatabase.jsonfiles.Automatic line breakIf you don't want anymore an extra line break at the end of printed strings, you can turn off this functionality:[task]
# turn off auto line break
config AutoLineBreak=0
: `Hello `
: `World`
# turn on auto line break
config AutoLineBreak=1
: Hello WorldLinesYou can draw lines with the characters=or-. If you pick one, only this one is allowed to appear on the same line.[task1]

$ program1
$ program2

-----------------

[task2]
pass

=================DemoThe demo is arepositorythat contains abackstage.tasksfile similar to the one used to build, package and publish my projects. Your mission, if you accept it, is to clone the demo repository and run thebackstage.taskswhich contains tasks to create a newPythonHello Friend !project, build it, perform versioning, initGit, performGitCommit andGitPush, and even push the latest built package toPyPI!# 1- clone the repository$gitclonehttps://github.com/pyrustic/project
$cdproject# 2- install backstage$pipinstallbackstage# 3- install buildver$pipinstallbuildver# 4- install setupinit$pipinstallsetupinit# 5- list the tasks available in the `backstage.tasks` file$backstage-c
Availabletasks(11):buildcheckcleangendocgitcommitgitinitgitpushinitreleasetestupload2pypi# 6- descriptive list of tasks$backstage-C# 7- initialize the project$backstageinit
Successfullyinitialized!# 8- run the project$python3-mproject
HelloFriend!# 9- build the project$backstagebuild
buildingv0.0.1...
Successfullybuilt'project'v0.0.1!
VERSIONfileupdatedfrom0.0.1to0.0.2# 10- check the project$backstagecheck
projectv0.0.2(source).whlv0.0.1(package)built28secsago# 11- initialize Git$backstagegitinit
Origin:https://github.com/pyrustic/project.git# 12- perform a Git Commit$backstagegitcommit# 13- perform a Git Push$backstagegitpush# 14- upload to PyPI$backstageupload2pypiNote:Commands9,12,13, and14can be replaced with one command:backstage releaseInstallationBackstageiscross platformand versions under1.0.0will be consideredBetaat best. It is built onUbuntuwithPython 3.8and should work onPython 3.5ornewer.For the first time$pipinstallbackstageUpgrade$pipinstallbackstage--upgrade--upgrade-strategyeagerBack to top