qitangle 0.5.5
QiTangle - IPython magic to tangle (quarto) code cells into python source files
Requires Python: >=3.12
Keywords: ipython, magic, literate, programming, scientific, markdown, reports, documentation

# QiTangle - IPython magic to entangle code from notebooks.


> export code from notebook cells to python source files

The first Quarto document I created was used to implement a python
module based on some math formula’s from an article (about measuring sap
flux in tree stems). Quarto let me follow the article and methodically,
step by step functions were build and documented along the formula’s and
some example calculations. This worked like a charm. When everything was
implemented and working all there was left to do was copy the code from
the document into a separate python file for use in our runtime
environment.

Unfortunately, any changes in eihter of the two sources - the notebook
and the script - had to be copied from one to theother. What I needed
was a way to convert or export selected code cells into a python source
code file automatically and I found this in NBDev. NBDev exports your
code, builds the documentation and, if desired, packages the code and
uploads to PyPi.

Jupyter notebooks however are a uncomfortable to use in combination with
*git*, even with the clean-up tools provided by NBDev, many unneeded git
diffs were checked in and worse; merge conflicts can be a huge problem.

Something closer to using Quarto’s own markdown was needed, simply
extracting or exporting python sourcefiles from Quarto documents.
Quarto’s python support is based on IPython and Jupyter kernels, and
these offer an extension mechanism called *magics*. These extensions are
available in Jupyter and Quarto notebooks.

Not finding what I needed I wrote a magic to export code from notebook
cells to python source files and called it *QiTangle* after *Q*uarto,
*I*Python and *TANGLE*, the latter being the name of a component and
process in Knuth’s *Literate programming* system that was responsible
for *tangling* the computer code together.

#### QiTangle and literate programming

Quarto’s rich document creation and interactive computing extended with
the ability to export selected Python code cells into sourcecode file
are not entirely dissimilar to Knuth’s Literate programming paradigm.
WEB’s system allowed for describring pieces of computer code and
algorithms that could be combined with each other. QiTangle is more
restricted in that it only allows chunks of lines of code to be exported
to a python source file, without combining expressions. It does however,
unlike many similar notebook based systems, allow cells to be
*reordered* into the exported source file. As in Knuth’s system,
QiTangle allows disconnecting the *narrative order* that descibes the
code in an order suitable to explain it, from the neccesities imposed on
the *ordering of code* by the computer language.

## Installing

``` sh
pip install qitangle
```

## Usage

### Loading the extension

    import qitangle
    %reload_ext qitangle.entangle

Using `%reload_ext` will reset the cell collection even when a kernel
process remains active as is the case with Quarto

### Tangling

Tangling is the process of exporting the individual code cells to a file
in the order as specified by the *label* and *dependencies* attributes
of an entangled cell.

Each `%%entangle` invocation adds a cell to a collection, it’s *label*
and *dependencies* inform the `%tangle` which cells depend on which
other cells and thereby ultimately the order in which they are exported.
By exporting it’s dependencies before the cell we ensure that prior to
excution of that cell everything, well it’s code dependencies at least,
are in place.

When updating results in a collection that can not be ordered correctly
cell execution is delayed until their dependencies are added.

Optionally each cell individually can be reformatted through *autopep8*
and optionally when exporting, the entire produced file can be
reformatted. Configuration of autopep8 is described in [it’s
manual](https://pypi.org/project/autopep8/#configuration "autopep8 configuration on pypi").

### Invocations

#### Line level magic

Line magic `%tangle` controls the tangle

With the first invocation the target file is set

    %tangle [-l libfolder] modulepath

With subsequent invocations various commands are given

    %tangle [-r] [-e] [-m] [-s] [--stdlib] [labels]

#### Cell level magic

Cell magic `%%entangle` updates the code collection

    %%entangle [-f] label [dependencies]

### Options

The options for `%tangle`’s first invocation:

- `-f`, `--format`: Always re*format* the tangled file with autopep8
  after export
- `-l`, `--lib`: changes the root for module paths - also useful for
  generating tests

For subsequent invocations of `%tangle`:

- `-e`, `--export`: export ordered cells to designated file
- `-f`, `--format`: re*format* the tangled file with autopep8 after
  export
- `-r`, `--report`: report missing dependecies and unused cells
- `-m`, `--missing`: list missing dependencies for delayed cells
- `-s`, `--stdlib`: adds all currently missing stdlib dependencies

Options for `%%entangle` are:

- `-f`, `--format`: re*format* code in cell with autopep8

### Required positional arguments:

For the first invocation of `%tangle`

- `module-path`:= The module-path for the export

For `%%entangle`

- `label` := The label for a cell - dependencies refer to these

### Optional positional arguments:

For the first invocation of `%tangle`

- `libfolder` := A parent folder for the module-path, directly under
  project-root or project-root/src

For subsequent invocations of `%tangle`:

- `labels` := The labels of cells to be exported / reported on

*Note* that when using `labels` with `--export` dependencies are not
verified.

For `%%entangle`

- `dependencies`:= The labels of dependencies for the code cell

### Argument format

    label = name
    labels = *1(name *(SPACE name))
    dependencies = *1(name *(SPACE name))
    modulepath = name *(DOT name)
    name = 1*TEXTDATA

### Discussion

A *module-path* looks pretty much the same as in a python *import*
statement, the *cell-label* is a name for the cell within the collection
of cells - or python file - specified by the *module-path* and the
*dependencies* are the labels of other cells within the specified
collection upon which this cell depends and which are exported before
this cell, if possible.

When an inconsistency is found in the dependency information the export
will fail with a warning and nothing will be (over-) written. Fixing the
consistency in the dependicies is the only remedy.

The code in the cell will only run when the label and dependencies of
the cell result in a valid code sequence. If not, cell execution is
delayed until that cell’s dependencies have become consistent again.

Reformatting code can be done for individual cells when updating them,
as well as the entire exported file. Reformatting is done by *autopep8*.

## Additional documentation

You can view the documentation local with

``` sh
quarto preview srcdocs/index.qmd
```

Or (with a local .venv active) render with

``` sh
quarto render docs
```

and use them from from `srcdocs/_site`

> Note: rendering and previewing will also rebuild code in src/