Package: codebraid
Codebraid – live code in Pandoc MarkdownCodebraid is a Python program that enables executable code inPandocMarkdown documents.  Using Codebraid can be as
simple as adding a class to your code blocks' attributes, and then runningcodebraidrather thanpandocto convert your document from Markdown to
another format.codebraidsupports almost all ofpandoc's options and
passes them topandocinternally.  SeeCodebraid Preview for VS Codefor editor support.  See theCodebraid websitefor
additional examples and documentation.Codebraid provides two options for executing code.  It includes a built-in
code execution system that currently supportsPython 3.7+,Julia,Rust,R,Bash,JavaScript,GAPandSageMath.  Code can also be
executed usingJupyter kernels, with support for rich output like plots.Development:https://github.com/gpoore/codebraidCiting Codebraid:"Codebraid: Live Code in Pandoc Markdown",
Geoffrey M. Poore,Proceedings of the 18th Python in Science Conference,2019, 54-61.View example HTML output, or see the Markdown source or raw HTML (the Python
and Rust examples demonstrate more advanced features at the end):Python example[Pandoc Markdown source]
[raw HTML]Jupyter example[Pandoc Markdown source]
[raw HTML]Rust example[Pandoc Markdown source]
[raw HTML]Julia example[Pandoc Markdown source]
[raw HTML]R example[Pandoc Markdown source]
[raw HTML]Bash example[Pandoc Markdown source]
[raw HTML]JavaScript example[Pandoc Markdown source]
[raw HTML]GAP example[Pandoc Markdown source]
[raw HTML]Simple exampleMarkdown sourcetest.md:```{.python .cb-run}
var = 'Hello from Python!'
var += ' $2^8 = {}$'.format(2**8)``````{.python .cb-run}print(var)```Runcodebraid(to save the output, add something like-o test_out.md, and
add--overwriteif it already exists):codebraidpandoc--frommarkdown--tomarkdowntest.mdOutput:Hello from Python! $2^8 = 256$As this example illustrates, variables persist between code blocks; by
default, code is executed within a single session.  Code output is also cached
by default so that code is only re-executed when modified.FeaturesComparison withJupyter,knitr, andPweaveCodebraidJupyter NotebookknitrPweavemultiple programming languages per document✓✓*✓†✓*multiple independent sessions per language✓inline code execution within paragraphs✓✓✓no out-of-order code execution✓✓‡✓no markdown preprocessor or custom syntax✓✓minimal diffs for easy version control✓✓✓insert code output anywhere in a document✓✓can divide code into incomplete snippets✓✓✓support for literate programming✓✓compatible with any text editor✓✓✓* One primary language from the Jupyter kernel.  The IPython kernel
supports additional languages via%%scriptmagics.  There is no continuity
between%%scriptcells, because each cell is executed in a separate process.
Some magics, such as those provided byPyJuliaandrpy2, provide more advanced capabilities.† knitr only provides continuity between code chunks for R, and more recently
Python and Julia.  Code chunks in other languages are executed individually
in separate processes.‡ Out-of-order execution is possible with R Markdown notebooks.The table above summarizes Codebraid features in comparison with Jupyter
notebooks (without extensions), knitr (R Markdown), and Pweave, emphasizing
Codebraid's unique features.  Here are some additional points to consider:Jupyter notebooks— Notebooks have a dedicated, browser-based graphical user
interface.  Jupyter kernels typically allow the code in a cell to be executed
without re-executing any preceding code, providing superior interactivity.
Codebraid has advantages for projects that are more focused on creating a
document than on exploratory programming.knitr— R Markdown documents have a dedicated user interface in R Studio.
knitr provides superior support for R, as well as significant Python and Julia
support that includes R integration.  Codebraid offers continuity between code
chunks for all supported languages, as well as multiple independent sessions
per language.  It also provides unique options for displaying code and its
output.More about key featuresEasy debugging— By default, stderr is shown automatically in the document
whenever there is an error, right next to the code that caused it.  It is also
possible to monitor code output in real time during execution via--live-output.Simple language support— Codebraid supports Jupyter kernels.  It also has a
built-in system for executing code.  Adding support for a new language with
this system can take only a few minutes.  Just create a config file that tells
Codebraid which program to run, which file extension to use, and how to write
to stdout and stderr.  Seelanguages/for examples.No preprocessor— Unlike many approaches to making code in Markdown
executable, Codebraid is not a preprocessor.  Rather, Codebraid acts on the
abstract syntax tree (AST) that Pandoc generates when parsing a document.
Preprocessors often fail to disable commented-out code blocks because the
preprocessor doesn't recognize Markdown comments.  Preprocessors can also fail
due to the finer points of Markdown parsing.  None of this is an issue for
Codebraid, because Pandoc does the Markdown parsing.No custom syntax— Codebraid introduces no additional Markdown syntax.
Making a code block or inline code executable uses Pandoc's existing syntax
for defining code attributes.Installation and requirementsInstallation:pip3 install codebraidorpip install codebraidManual installation:python3 setup.py installorpython setup.py installRequirements:Pandoc2.4+ (2.17.1.1+ recommended forcommonmark_x).Python 3.7+ withsetuptools, andbespon0.6
(besponinstallation is typically managed bypip/setup.py)For Jupyter support,jupyter_clientand language kernelsFor YAML metadata support,ruamel.yaml(can beruamel_yamlfor Anaconda installations)Converting a documentSimply runcodebraid pandoc <normal pandoc options>.  Codebraid currently
supports Pandoc Markdown (--from markdown) and CommonMark with Pandoc
extensions (--from commonmark_x) as input formats.Note that--overwriteis required to overwrite existing files.  If you are
using a defaults file,--from,--to, and--outputmust be given
explicitly and cannot be inherited from the defaults file.  If you are using a
defaults file and converting to a standalone Pandoc Markdown document,--standaloneshould be given explicitly rather than being inherited from the
defaults file.codebraidshould typically be run in the same directory as the document, so
that the default working directory for code is the document directory.If you are converting from Pandoc Markdown to Pandoc Markdown with--standalone(basically usingcodebraidto preprocess Markdown documents),
note that the following YAML metadata fields and command-line options are
ignored in that situation:header-includesand--include-in-headerinclude-beforeand--include-before-bodyinclude-afterand--include-after-bodytoc/table-of-contentsand--toc/--table-of-contentsThis is typically what you want.  Usually, "include" and a table of contents
are desired in a final output format like HTML or PDF, not in a Pandoc
Markdown file.  In the rare cases where "includes" and a table of contents are
needed in Markdown documents, this can be accomplished by piping the output ofcodebraidthroughpandoc.Additional non-Pandoc command-line options--live-output— Show code output (stdout and stderr) live in the terminal
during code execution.  For Jupyter kernels, also show  errors and a summary
of rich output.  Output still appears in the document as normal.Individual sessions can override this by settinglive_output=falsein the
document.--no-execute— Disables code execution.  Only use available cached output.--only-code-output={format} — Write code output in JSON Lines format to
stdout as soon as it is available, and do not create a document.This is intended for use with Codebraid Preview, so that document previews
can be updated during code execution.  Currently, the only supported format
iscodebraid_preview.  One JSON data object followed by a newline is
written to stdout for each code chunk.  In some cases, the data for a chunk
will be resent later if the data relevant for a chunk changes (for example,
if code execution fails after the first chunk runs, but in such a way that
an error message needs to be attached to the first chunk).  Data for a chunk
is sent as soon as it is available from code processing, from cache, or from
code execution (as soon as the chunk completes, typically before the session
completes).  Additional JSON data may be sent to provide tracking of code
execution progress or information such as metadata.  The JSON data provided
for formatcodebraid_previewmay change between minor versions.CachingBy default, code output is cached, and code is only re-executed when it is
modified.  The default cache location is a_codebraiddirectory in the
working directory (directory wherecodebraidis run, typically the document
directory).  This can be modified using--cache-dir.  Multiple documents can
share a single cache location.  A cache directory can be synced between
different operating systems (such as Windows and Linux) while retaining full
functionality so long as documents are in equivalent locations under the
user's home directory (as resolved byos.path.expanduser()).When multiple documents share the same cache location, each document will
automatically clean up its own unused, outdated files.  However, if a document
is deleted or renamed, it may leave behind unused files in the cache, so it
may be worth manually deleting and regenerating the cache in those
circumstances.  Future cache enhancements should be able to detect all unused
files, making this unnecessary.If you are working with external data that changes, you should runcodebraidwith--no-cacheor delete the cache as necessary to prevent the cache from
becoming out of sync with your data.  Future releases will allow external
dependencies to be specified so that caching will work correctly in these
situations.YAML metadataSome document-wide settings can be given in the Markdown YAML metadata.
Codebraid settings must be under either acodebraidorcodebraid_key in
the metadata.  Pandoc will ignorecodebraid_so it will not be available to
filters; this distinction should not typically be important.To use Jupyter kernels automatically for all sessions, simply setjupyter: true.  For example,---
codebraid:
  jupyter: true
---It is also possible to set a default kernel and/or default timeout.  For example,---
codebraid:
  jupyter:
    kernel: python3
    timeout: 120
---A Jupyter kernel and/or timeout can still be set in the first code chunk
for a given session, and will override the document-wide default.It is also possible to setlive_output: <bool>in the metadata.
Additional metadata settings will be added in future releases.Code optionsCommands (Classes)Code is made executable by adding a Codebraid class to itsPandoc attributes.
For example,`code`{.python}becomes`code`{.python .cb-run}.When code is executed, the output will depend on whether the built-in code
execution system or a Jupyter kernel is used.When code is executed with the built-in system, the output is equivalent to
collecting all code for each session of each language, saving it to a file,
and then executing it (with an added compile step for some languages).  For
example, running Python code is equivalent to saving it tofile.pyand then
runningpython file.py, while running R code is equivalent to saving it tofile.Rand then runningRscript file.R.  Code is not executed as it would
be in an interactive session (like runningpythonorRat the command
prompt).  As a result, some output that would be present in an interactive
session is absent.  For example, in interactive sessions for some languages,
simply entering a variable returns a string representation without explicit
printing, and plotting opens a separate image window or displays an image
inline.  Such output is absent in Codebraid unless it is also produced when
code is executed as a script rather than in an interactive session.  The.cb-exprcommand is provided for when an inline string representation of a
variable is desired.An option for interactive-style code execution with the built-in system is
planned for a future release.  In the meantime, many interactive-style
features are available between the.cb-exprcommand and Jupyter kernels.When code is executed with a Jupyter kernel, the default output will be
equivalent to executing it in a Jupyter notebook.  Rich output such plots,
images, and LaTeX math will be displayed automatically by default.  This can
be customized by using theshowandhideoptions.All classes for making code executable are listed below.  These all have the
form.cb-<command>.  Classes with the form.cb.<command>(period rather
than hyphen) are supported for Pandoc Markdown (--from markdown), but not
forcommonmark_xsince it has a more restricted class syntax.  The forms
shown below (.cb-<command>) should be preferred for compatibility across
Markdown variants supported by Pandoc..cb-code— Insert code verbatim, but do not run it.  This is primarily
useful when combined with other features like naming and then copying code
chunks..cb-expr— Evaluate an expression and interpret the result as Markdown.
Only works with inline code.  This is not currently compatible with Jupyter
kernels..cb-nb— Execute code in notebook mode.  For inline code, this is
equivalent to.cb-exprwith verbatim output unless a Jupyter kernel is
used, in which case rich output like plots or LaTeX will be displayed.  For
code blocks, this inserts the code verbatim, followed by any printed output
(stdout) verbatim.  If stderr exists, it is also inserted verbatim.  When a
Jupyter kernel is used, rich output like plots or LaTeX is also displayed..cb-paste— Insert code and/or output copied from one or more named code
chunks.  Thecopykeyword is used to specify chunks to be copied.  This
does not execute any code.  Unlessshowis specified, display options are
inherited from the first copied code chunk.If content is copied from multiple code chunks that are executed, all code
chunks must be in the same session and must be in sequential order without
any omitted chunks.  This ensures that what is displayed is always
consistent with what was executed.If content is copied from anothercb-pastecode chunk, only a single code
chunk can be copied.  This reduces the indirection that is possible when
displaying the output of code that has been executed.  This restriction may
be removed in the future..cb-run— Run code and interpret any printed content (stdout) as Markdown.
Also insert stderr verbatim if it exists.  When a Jupyter kernel is used,
rich output like plots or LaTeX is also displayed.Keyword argumentsPandoc code attribute syntax allows keyword arguments of the formkey=value,
with spaces (notcommas) separating subsequent keys.valuecan be
unquoted if it contains only letters and some symbols; otherwise, double
quotation marks"value"are required.  For example,{.python key1=value1 key2=value2}Codebraid adds support for additional keyword arguments.  In some cases,
multiple keywords can be used for the same option.  This is primarily for
Pandoc compatibility.First chunk settingsThese are only permitted for the first code chunk in a session (or the first
chunk for a language, if a session is not specified and thus the default
session is in use).executable={string} — Executable to use for running or compiling code,
instead of the default.  This only applies to Codebraid's built-in code
execution system.executable_opts={string} — Command-line options passed toexecutable.
This only applies to Codebraid's built-in code execution system.args={string} — Command-line arguments passed to code during execution.
For example, this could be used to add values tosys.argvfor Python.
This only applies to Codebraid's built-in code execution system.jupyter_kernel={string} — Jupyter kernel to use for executing code instead
of Codebraid's built-in code execution system.  Multiple Jupyter kernels can
be used within a single document, and multiple sessions are possible per
kernel.  Except when otherwise specified, Jupyter kernels should be usable
just like the built-in code execution system.jupyter_timeout={int} — Jupyter kernel timeout per code chunk in seconds.
The default is 60.live_output={true,false} — Show code output (stdout and stderr) live
in the terminal during code execution.  For Jupyter kernels, also show
errors and a summary of rich output.  Output still appears in the document
as normal.  Showing output can also be enabled via the command-line option--live-output.Whenlive_output=falseis set for a session, this setting takes precedence
over the command-line option--live-output, and output will not be shown
for that session.All output is written to stderr, so stdout only contains the document when--outputis not specified.  Output is interspersed with delimiters marking
the start of each session and the start of each code chunk.  The delimiters
for the start of each code chunk include source names and line numbers.With Codebraid's built-in code execution system, the output for a code chunk
may be delayed until all code in the chunk has finished executing, unless
code output is line buffered or code manually flushes stdout and stderr.
For example, with Python you may want to use print functions likeprint("text", flush=True).  Another option is to use Python in
line-buffered mode by settingexecutable_opts="-u"in the first code chunk
of a session.With Jupyter kernels, the output for a code chunk will be delayed until all
code in the chunk has finished executing.Executioncomplete={true,false} — By default, code chunks must contain complete
units of code (function definitions, loops, expressions, and so forth). Withcomplete=false, this is not required.  Any stdout from code chunks withcomplete=falseis accumulated until the next code chunk withcomplete=true(the default value), or until the end of the session,
whichever comes first.Settingcompleteis incompatible withoutside_main=true, since thecompletestatus of code chunks withoutside_main=trueis inferred
automatically.outside_main={true,false} — This allows code chunks to overwrite the
Codebraid template code when code is executed with Codebraid's built-in code
execution system.  It is primarily useful for languages like Rust, in which
code is inserted by default into amain()template.  In that case, if a
sessionstartswith one or more code chunks withoutside_main=true,
these are used instead of the beginning of themain()template.
Similarly, if a sessionendswith one or more code chunks withoutside_main=true, these are used instead of the end of themain()template.  If there are any code chunks in between that lackoutside_main(that is, defaultoutside_main=false), then these will have their stdout
collected on a per-chunk basis like normal.  Having code chunks that lackoutside_mainis not required; if there are none, the total accumulated
stdout for a session belongs to the last code chunk in the session.outside_main=trueis incompatible with explicitly settingcomplete.  Thecompletestatus of code chunks withoutside_main=trueis inferred
automatically.session={identifier-style string} — By default, all code for a given
language is executed in a single, shared session so that data and variables
persist between code chunks.  This option allows code to be separated into
multiple independent sessions.  Session names must be Python-style
identifiers.Displayfirst_number/startFrom/start-from/start_from={integer ornext} —
Specify the first line number for code when line numbers are displayed.nextmeans continue from the last code in the current session.hide={markup,copied_markup,code,stdout,stderr,expr,rich_output,all} — Hide some or all of the elements that are displayed
by default.  Elements can be combined.  For example,hide=stdout+stderr.
Note thatexpronly applies to.cb-expror.cb-nbwith inline code
using Codebraid's built-in code execution system, since only these evaluate
an expression.rich_outputis currently only relevant for Jupyter
kernels.hide_markup_keys={key(s)} — Hide the specified code chunk attribute key(s)
in the Markdown source displayed viamarkuporcopied_markup.  Multiple
keys can be specified viahide_markup_keys=key1+key2.hide_markup_keysonly applies to the code chunk in which it is used, to
determined themarkupfor that code chunk.  Thus, it only affectscopied_markupindirectly.line_numbers/numberLines/number-lines/number_lines={true,false}
— Number code lines in code blocks.show={markup,copied_markup,code,stdout,stderr,expr,rich_output,none} — Override the elements that are displayed by
default.expronly applies to.cb-exprand to.cb-nbwith inline code
using Codebraid's built-in code execution system, since only these evaluate
an expression.  Elements can be combined.  For example,show=code+stdout.Each element exceptrich_outputcan optionally specify a format fromraw,verbatim, orverbatim_or_empty.  For example,show=code:verbatim+stdout:raw.rawmeans interpreted as Markdown.verbatimproduces inline code or a code block, depending on context.
Nothing is produced if there is no content (for example, nothing in
stdout.)verbatim_or_emptyproduces inline code containing a single
non-breaking space or a code block containing a single empty line in the
event that there is no content.  It is useful when a placeholder is
desired, or a visual confirmation that there is indeed no output.Forrich_output, the format is specified as one or more abbreviations for
the mime types of the output to be displayed.  For example,rich_output:plainwill displaytext/plainoutput if it exists, and
otherwise nothing.rich_output:png|plainwill display a PNG image if it
exists, or otherwise will fall back to plain text if available.  The
following formats are currently supported:latex(corresponds totext/latex)html(text/html)markdown(text/markdown)plain(text/plain)png(image/png)jpgandjpeg(image/jpeg)svg(image/svg+xml)pdf(application/pdf)Forrich_outputformats with atext/*mime type (latex,html,markdown,plain), it is possible to specify whether they are displayedraw,verbatim, orverbatim_or_empty.  For example,show=rich_output:latex:rawandshow=rich_output:latex:verbatim.rawtreatslatexandhtmlas raw content with those formats embedded within
Markdown.rawtreatsmarkdownandplainas Markdown.  When a display
style is not specified, allrich_outputformats with atext/*mime type
are displayedrawby default, except forplainwhich is displayedverbatim.markupdisplays the Markdown source for the inline code or code block.
Because the Markdown source is not available in the Pandoc AST but rather
must be recreated from it, the Markdown source displayed withmarkupmay
use a different number of backticks, quote attribute values slightly
differently, or contain other insignificant differences from the original
document.copied_markupdisplays the Markdown source for code chunks copied viacopy.exprdefaults torawif a format is not specified.rich_outputdefaults tolatex|markdown|png|jpg|svg|plain.  All others default toverbatim.example={bool} — Insert a code block containing the Markdown source of the
code chunk, followed by the rest of the output as normal.  This is only
valid for inline code if the code is in a paragraph by itself.  This option
is currently not compatible with--only-code-outputand Codebraid Preview.
This option is intended primarily for documentation about Codebraid.Copyingcopy={chunk name(s)} — Copy one or more named code chunks.  Whencopyis
used with a command like.cb-runthat executes code, only the code is
copied, and it is executed as if it had been entered directly.  Whencopyis used with.cb-code, only the code is copied and nothing is executed.
Whencopyis used with.cb-paste, both code and output are copied, and
nothing is executed.  Multiple code chunks may be copied; for example,copy=name1+name2.  In that case, the code from all chunks is concatenated,
as is any output that is copied.  Becausecopybrings in code from other
code chunks, the actual content of a code block or inline code usingcopyis discarded.  As a result, this must be empty, or a space or underscore can
be used as a placeholder.name={identifier-style string} — Name a code chunk so that it can later be
copied by name.  Names must be Python-style identifiers.Including external filesinclude_file={path} — Include the specified file.  A leading~/or~<user>/is expanded to the user's home directory under all operating
systems, including under Windows with both slashes and backslashes.Wheninclude_fileis used with a command like.cb-runthat executes
code, the file is included and executed as part of the current session just
as if the file contents had been entered directly.  Wheninclude_fileis
used with.cb-code, the file is included and displayed just as if it had
been entered directly.  Becauseinclude_filebrings in code from another
file, the actual content of a code block or inline code usinginclude_fileis discarded.  As a result, this must be empty, or a space or underscore can
be used as a placeholder.include_encoding={encoding} — Encoding for included file.  The default
encoding is UTF-8.include_lines={lines/line ranges} — Include the specified lines or line
ranges.  For example,1-3,5,7-9,11-.  Line numbers are one-indexed.  Line
ranges are inclusive, so1-3is1up to and including3.  If a range
ends with a hyphen, like11-, then everything is included from the line
through the end of the file.Cannot be combined with otherincludeoptions that specify what is to
be included.include_regex={regex} — Include the first segment of the file that matches
the provided regular expression.Keep in mind that Pandoc's key-value attributes evaluate backslash escapes
in values whether or not the values are quoted with double quotation marks,
so two levels of backslash-escaping are always necessary (one for Pandoc's
strings, one for the regex itself; there are no raw strings).  Regular
expressions usemultiline mode, so^/$match the start/end of a line,
and\A/\Zcan be used to match the start/end of the file.  Regular
expressions usedotall mode, so.matches anything including the newline\n; use[^\n]when this is not desired.Cannot be combined with otherincludeoptions that specify what is to
be included.include_start_string={string} — Include everything from the first
occurrence of this string onward.Can only be combined with otherincludeoptions that specify the end of
what is to be included.include_start_regex={regex} — Include everything from the first
match of this regex onward.Can only be combined with otherincludeoptions that specify the end of
what is to be included.  Seeinclude_regexfor notes on regex usage.include_after_string={string} — Include everything after the first
occurrence of this string onward.Can only be combined with otherincludeoptions that specify the end of
what is to be included.include_after_regex={regex} — Include everything after the first
match of this regex onward.Can only be combined with otherincludeoptions that specify the end of
what is to be included.  Seeinclude_regexfor notes on regex usage.include_before_string={string} — Include everything before the first
occurrence of this string.Can only be combined with otherincludeoptions that specify the start of
what is to be included.  If the start is specified, then the first
occurrence after this point is used, rather than the first occurrence in the
overall file.include_before_regex={regex} — Include everything before the first
match of this regex.Can only be combined with otherincludeoptions that specify the start of
what is to be included.  If the start is specified, then the first match
after this point is used, rather than the first match in the overall file.
Seeinclude_regexfor notes on regex usage.include_end_string={string} — Include everything through the first
occurrence of this string.Can only be combined with otherincludeoptions that specify the start of
what is to be included.  If the start is specified, then the first
occurrence after this point is used, rather than the first occurrence in the
overall file.include_end_regex={regex} — Include everything through the first
match of this regex.Can only be combined with otherincludeoptions that specify the start of
what is to be included.  If the start is specified, then the first match
after this point is used, rather than the first match in the overall file.
Seeinclude_regexfor notes on regex usage.