Package: python-pake
About pakepake is a make-like python build utility where tasks, dependencies and
build commands can be expressed entirely in python, similar to ruby
rake.pake supports automatic file/directory change detection when dealing with
task inputs and outputs, and also parallel builds.pake requires python3.5+This readme contains only a little information about how to use pake, please
checkout the latest documentation at the links above for an expansive
overview on how pake works and how pakefiles should be written.InstallingNote: pake is Alpha and likely to change some.To install the latest release use:sudo pip3 installpython-pake--upgradeIf you want to install the development branch you can use:sudo pip3 installgit+git://github.com/Teriks/pake@developExample project using pakeI am using libasm_io to help test pake and have included a pakefile
build along side the makefiles in that project.https://github.com/Teriks/libasm_ioSee the build with pake section oflibasm_io/build-readme.mdhere.Writing Basic TasksHere’s a contrived pake demo:importpake# Tasks are registered the the pake.Pake object# returned by pake's initialization call, using the task decorator.pk=pake.init()# Try to grab a command line define.# In particular the value of -D CC=..# CC will default to 'gcc' in this case if# it was not specified.CC=pk.get_define('CC','gcc')# you can also use the syntax: pk["CC"] to# attempt to get the defines value, if it is not# defined then it will return None.# ===# If you just have a single input/output, there is no# need to pass a list to the tasks inputs/outputs@pk.task(i='foo/foo.c',o='foo/foo.o')deffoo(ctx):# Execute a program (gcc) and print its stdout/stderr to the tasks output.# ctx.call can be passed a command line as variadic arguments, an iterable, or# as a string.  It will automatically flatten out non string iterables in your variadic# arguments or iterable object, so you can pass an iterable such as ctx.inputs# as part of your full command line invocation instead of trying to create the command# line by concatenating lists or using the indexer on ctx.inputs/ctx.outputsctx.call(CC,'-c',ctx.inputs,'-o',ctx.outputs)# Pake can handle file change detection with multiple inputs# and outputs. If the amount of inputs is different from# the amount of outputs, the task is considered to be out# of date if any input file is newer than any output file.# When the amount of inputs is equal to the amount of outputs,# pake will compare each input to its corresponding output# and collect out of date input/outputs into ctx.outdated_inputs# and ctx.outdated_outputs respectively.  ctx.outdated_pairs# can be used to get a generator over (input, output) pairs,# it is shorthand for zip(ctx.outdated_inputs, ctx.outdated_outputs)@pk.task(i=pake.glob('bar/*.c'),o=pake.pattern('bar/%.o'))defbar(ctx):# zip together the outdated inputs and outputs, since they# correspond to each other, this iterates of a sequence of python# tuple objects in the form (input, output)fori,oinctx.outdated_pairs:ctx.call(CC,'-c',i,'-o',o)# This task depends on the 'foo' and 'bar' tasks, as# specified with the decorators leading parameters.# It outputs 'bin/baz' by taking the input 'main.c'# and linking it to the object files produced in the other tasks.@pk.task(foo,bar,o='bin/baz',i='main.c')defbaz(ctx):"""Use this to build baz"""# Documentation strings can be viewed by running 'pake -ti' in# the directory the pakefile exists in, it will list all documented# tasks with their python doc strings.# The pake.FileHelper class can be used to preform basic file# system operations while printing information about the operations# it has completed to the tasks output.file_helper=pake.FileHelper(ctx)# Create a bin directory, this won't complain if it exists alreadyfile_helper.makedirs('bin')# ctx.dependency_outputs contains a list of all outputs that this# tasks immediate dependencies producectx.call(CC,'-o',ctx.outputs,ctx.inputs,ctx.dependency_outputs)@pk.taskdefclean(ctx):"""Clean binaries"""file_helper=pake.FileHelper(ctx)# Clean up using the FileHelper object.# Remove the bin directory, this wont complain if 'bin'# does not exist.file_helper.rmtree('bin')# Glob remove object files from the foo and bar directoriesfile_helper.glob_remove('foo/*.o')file_helper.glob_remove('bar/*.o')# Run pake; The default task that will be executed when# none are specified on the command line will be 'baz' in# this case.# The tasks parameter is optional, but if it is not specified# here, you will be required to specify a task or tasks on the# command line.pake.run(pk,tasks=baz)Output from commandpake:=====Executingtask:"bar"gcc-cbar/bar.c-obar/bar.o=====Executingtask:"foo"gcc-cfoo/foo.c-ofoo/foo.o=====Executingtask:"baz"CreatedDirectory(s):"bin"gcc-obin/bazmain.cfoo/foo.obar/bar.oOutput from commandpake clean:=====Executingtask:"clean"RemovedDirectory(s):"bin"GlobRemovedFiles:"foo/*.o"GlobRemovedFiles:"bar/*.o"Concurrency Inside TasksWork can be submitted to the threadpool pake is running its tasks on to achieve a
predictable level of concurrency for sub tasks that is limited by the–jobscommand line argument,
or thejobsparameter ofpake.runandpake.Pake.run.Example:importpake# functools.partial is used for binding argument values to functionsfromfunctoolsimportpartialpk=pake.init()@pk.task(i=pake.glob('src/*.c'),o=pake.pattern('obj/%.o'))defbuild_c(ctx)file_helper=pake.FileHelper(ctx)# Make 'obj' directory if it does not exist.# This does not complain if it is already there.file_helper.makedirs('obj')# Start multitaskingwithctx.multitask()asmt:fori,oinctx.outdated_pairs:# Read the section 'Output synchronization with ctx.call & ctx.subpake'# in the 'Concurrency Inside Tasks` page on http://pake.readthedocs.io# for an explanation of 'sync_call' below, and how output# synchronization is achieved for ctx.call and ctx.subpakesync_call=partial(ctx.call,collect_output=pk.max_jobs>1)# Submit a work function with arguments to the threadpoolmt.submit(sync_call,['gcc','-c',i,'-o',o])@pk.task(build_c,i=pake.glob('obj/*.o'),o='main')defbuild(ctx):# Utilizing the automatic non string iterable# flattening here to pass ctx.inputs and ctx.outputsctx.call('gcc',ctx.inputs,'-o',ctx.outputs)pake.run(pk,tasks=build)Running Sub PakefilesPake is able to run itself through the use ofpake.TaskContext.subpakeandpake.subpake.pake.subpakeis meant to be used outside of tasks, and can even be
called before pake is initialized.pake.TaskContext.subpakeis preferred for use inside of tasks because
it handles writing to the task’s output queue for you, without having to specify
extra parameters topake.subpaketo get it working correctly.pake.TaskContextinstance is passed into the single argument of each task function,
which you can in turn callsubpakefrom.Defines can be exported to pakefiles ran with thesubpakefunctions usingpake.export.pake.subpakeandpake.TaskContext.subpakeuse the–stdin-definesoption of
pake to pass exported define values into the new process instance, which means you can overwrite your
exported define values with-D/–definein the subpake command arguments if you need to.Export / Subpake Example:importpakepk=pake.init()# Try to get the CC define from the command line,# default to 'gcc'.CC=pk.get_define('CC','gcc')# Export the CC variable's value to all invocations# of pake.subpake or ctx.subpake as a define that can be# retrieved with pk.get_define()pake.export('CC',CC)# You can also export lists, dictionaries sets and tuples,# as long as they only contain literal values.# Literal values being: strings, integers, floats; and# other lists, dicts, sets and tuples.  Collections must only# contain literals, or objects that repr() into a parsable literal.pake.export('CC_FLAGS',['-Wextra','-Wall'])# Nesting works with composite literals,# as long as everything is a pure literal or something# that str()'s into a literal.pake.export('STUFF',['you',['might',('be',['a',{'bad':['person',['if',{'you','do'},('this',)]]}])]])# This export will be overrode in the next callpake.export('OVERRIDE_ME',False)# Execute outside of a task, by default the stdout/stderr# of the subscript goes to this scripts stdout.  The file# object to which stdout gets written to can be specified# with pake.subpake(..., stdout=(file))# This command also demonstrates that you can override# your exports using the -D/--define optionpake.subpake('sometasks/pakefile.py','dotasks','-D','OVERRIDE_ME=True')# This task does not depend on anything or have any inputs/outputs# it will basically only run if you explicitly specify it as a default# task in pake.run, or specify it on the command line@pk.taskdefmy_phony_task(ctx):# Arguments are passed in a variadic parameter...# Specify that the "foo" task is to be ran.# The scripts output is written to this tasks output queuectx.subpake('library/pakefile.py','foo')# Run this pake script, with a default task of 'my_phony_task'pake.run(pk,tasks=my_phony_task)Output from the example above:***entersubpake[1]:pake[1]:EnteringDirectory"(REST OF PATH...)/paketest/sometasks"=====ExecutingTask:"dotasks"DoTaskspake[1]:ExitingDirectory"(REST OF PATH...)/paketest/sometasks"***exitsubpake[1]:=====ExecutingTask:"my_phony_task"***entersubpake[1]:pake[1]:EnteringDirectory"(REST OF PATH...)/paketest/library"=====ExecutingTask:"foo"Foo!pake[1]:ExitingDirectory"(REST OF PATH...)/paketest/library"***exitsubpake[1]:Running pakecdyour_pakefile_directory# Run pake with up to 10 tasks running in parallelpake-j10pake will look for “pakefile.py” or “pakefile” in the current directory
and run it.Or you can specify one or more files to run with-f/–file. The
switch does not have multiple arguments, but it can be used more than
once to specify multiple files.For example:pake-fpakefile.py foopake-fyour_pakefile_1.py-fyour_pakefile_2.py fooYou can also specify multiple tasks, but do not rely on unrelated tasks
being executed in any specific order because they won’t be. If there is
a specific order you need your tasks to execute in, the one that comes
first should be declared a dependency of the one that comes second, then
the second task should be specified to run.When running parallel builds, leaf dependencies will start executing
pretty much simultaneously, and non related tasks that have a dependency
chain may execute in parallel.pake task unrelated_task order_independent_phonyCommand Line Optionsusage: pake [-h] [-v] [-D DEFINE] [--stdin-defines] [-j JOBS] [-n]
            [-C DIRECTORY] [-t] [-ti] [--sync-output {True, False, 1, 0}]
            [-f FILE]
            [tasks [tasks ...]]

positional arguments:
  tasks                 Build tasks.

optional arguments:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit
  -D DEFINE, --define DEFINE
                        Add defined value.
  --stdin-defines       Read defines from a Python Dictionary piped into
                        stdin. Defines read with this option can be
                        overwritten by defines specified on the command line
                        with -D/--define.
  -j JOBS, --jobs JOBS  Max number of parallel jobs. Using this option enables
                        unrelated tasks to run in parallel with a max of N
                        tasks running at a time.
  -n, --dry-run         Use to preform a dry run, lists all tasks that will be
                        executed in the next actual invocation.
  -C DIRECTORY, --directory DIRECTORY
                        Change directory before executing.
  -t, --show-tasks      List all task names.
  -ti, --show-task-info
                        List all tasks along side their doc string. Only tasks
                        with doc strings present will be shown.
  --sync-output {True, False, 1, 0}
                        Tell pake whether it should synchronize task output
                        when running with multiple jobs. Console output can
                        get scrambled under the right circumstances with this
                        turned off, but pake will run slightly faster. This
                        option will override any value in the PAKE_SYNC_OUTPUT
                        environmental variable, and is inherited by subpake
                        invocations unless the argument is re-passed with a
                        different value or overridden in pake.init.
  -f FILE, --file FILE  Pakefile path(s). This switch can be used more than
                        once, all specified pakefiles will be executed in
                        order with the current directory as the working
                        directory (unless -C is specified).