texsurgery 0.6.3
Replace some commands and environments within a TeX document by evaluating code inside a jupyter kernel
Requires Python: >=3.6

# TexSurgery

Replaces some commands and environments within a TeX document by evaluating code inside a jupyter kernel.

Much like [sagetex](https://github.com/sagemath/sagetex), but with the following differences:

 1. `sagetex` collects all the code using LaTeX and only then runs `sage` to get the LaTeX output, which definitely works, but this conflicts with some interesting LaTeX packages and is slower than a direct conversion.
 2. `TexSurgery` works in any language with a jupyter kernel. In particular, you don't need to install `sagemath`.

## Installation

### Install python package

    `python3 -m pip install texsurgery`

### Install tex library

This is usually not required, but recommended. The instructions are different for `MiKTeX` and `TeX Live`:

    - `TeX Live`: Install the package with `tlmgr --verify-repo=none install texsurgery`
    - `MiKTeX`: Use the graphical interface.
    - _If the above doesn't work_: Install the CTAN LaTeX package manually: `https://ctan.org/pkg/texsurgery`
    - _If all of the above still don't work_: Copy the file `texsurgery.sty` from the folder `texsurgery/tex/texsurgery` into the current working folder where your `tex` document lies.

### Installation from source

    git clone https://framagit.org/pang/texsurgery.git
    cd texsurgery
    python3 -m pip install -U .

## Basic usage

The most simple use case is to insert a jupyter kernel output in a latex document. In order to do so, the document must include a line like

```
\usepackage[python3]{texsurgery}
```

to tell `texsurgery` which kernel to use. Then texsurgery can perform the following substitutions:

### Environments

#### run

Code blocks like

```
\begin{run}
code
\end{run}
```

will pass `code` to the kernel, and then be replaced by the output (both the input and the output may consist on several lines).

#### runsilent

Works like `run` but no output is shown.

#### eval

allows to make a substitution of a single line input in the middle of a line. For example

```
The result of 2+2 is \eval{2+2}.
```

will be replaced to

```
The result of 2+2 is 4.
```

##### Formatting

Optional parameters `type` and `format` can be passed to `eval` in order to format the result.

`type` can be any python type, plus `string` and `tex`. The output is converted to the corresponding type (`string` works like `str`, if the outout is a string between quotes, the quotes are eliminated; `tex` comverts a latex expression with escaed charcters to its unescaped version).

`format` can be any python format string, or `upper` or `lower`.

If the type is not provided, it will be infered from the format if possible.

So for example, the following code

```
\eval[type=float,format=.2f]{3*5}
\eval[type=float,format=8.2f]{3*5}
\eval[format=upper]{'a'+'b'}
\eval[type=string,format=upper]{'a'+'b'}
\eval[type=tex]{'\\frac{3}{5}'}
```

will result in

```
15.00
   15.00
'AB'
AB
\frac{3}{5}
```

#### evalstr

#### evaltex

#### sage

#### sinput

#### sif

#### srepl

## Command line usage

You can call it from the command line as follows:

```
texsurgery input_file.tex -o output_file.tex
```

will perform the code substitutions in `input_file.tex` and write the result in `output_file.tex`

The possible parameters are:

- `-h, --help` shows the help message and exits,
- `input_file` the file to read from. If none given, the standard input will be used.
- `--output_file output_file, -o output_file` writes the result in output file. If it is not provided, the result is directed to the standard output
- `-tex` just perform the code substitution in the latex content (this is the default)
- `-pdf` create a pdf file from the resulting modified tex. If the `-o` option is not provided, texsurgery will try to guess the name of the resulting file. This option deppends on `pdflatex`being installed in the system.
- `--pdflatex-options` options to be passed to pdflatex (it requires the `-pdf` option).

Since it can use standard input/output, it is pipe friendly. So, for example:

```
texsurgery input_file.tex | pandoc --from latex -o output_file.html
```
will perform the code substitutions and convert the result to a html (provided you have `pandoc` installed).

Likewise, if you have a markdown file `input.md` with the lines

```
\usepackage[python3]{texsurgery}

The result of multiplying 3 and 5 is \eval{3*5}
```

And run

```
pandoc input.md --from markdown --to latex -s | texsurgery
```

you will get the corresponding latex file with the `\usepackage` line removed, and the `\eval{3*5}` text substituted by the corresponding result `15`.

In fact, since texsurgery does not require fully correct `.tex` code, you can directly run texsurgery in the markdown file!

## Testing

If you installed from source, the following command will perform some common tests, and specific tests for some of the kernels that are installed:

    python3 -m unittest tests

## Selectors

### `find` and `findall`

`texsurgery` can also gather information using css-style selectors:

.. doctest::

    >>> from texsurgery.texsurgery import TexSurgery
    >>> tex = open('../tests/test_find.tex').read()
    >>> # An environment, which can be question or questionmultx,
    >>> # which contains an environment runsilent, and captures its content
    >>> TexSurgery(tex).findall('question,questionmultx runsilent')
    [('questionmultx', [('runsilent', '\na = randint(1,10)\n')]), ('questionmultx', [('runsilent', '\na = randint(2,10)\n')]), ('question', [('runsilent', '\na = randint(2,10)\nf = sin(a*x)\nfd = f.derivative(x)\n')])]
    >>> # An environment, which can be question or questionmultx,
    >>> # which contains an environment choices
    >>> # which contains a command \correctchoice, and captures its argument
    >>> TexSurgery(tex).findall('question,questionmultx choices \correctchoice{choice}')
    [('question', [('choices', [('\\correctchoice', {'choice': '$\\sage{fd}$'})])])]
    >>> # An environment questionmultx which contains a command
    >>> # \AMCnumericChoices with two arbitrary arguments
    >>> TexSurgery(tex).findall('questionmultx \AMCnumericChoices[_nargs=2]')
    [('questionmultx', [('\\AMCnumericChoices', {'arg0': '\\eval{8+a}', 'arg1': 'digits=2,sign=false,scoreexact=3'})]), ('questionmultx', [('\\AMCnumericChoices', {'arg0': '\\eval{8*a}', 'arg1': 'digits=2,sign=false,scoreexact=3'})])]
    >>> # Environment questionmultx, with first argument exactly equal to basic-multiplication
    >>> TexSurgery(tex).find(r'questionmultx{@basic-multiplication}')
    ('questionmultx', {'@basic-multiplication': 'basic-multiplication'}, '\n\\begin{runsilent}\na = randint(2,10)\n\\end{runsilent}\nWhat is $8*\\eval{a}$?\n\\AMCnumericChoices{\\eval{8*a}}{digits=2,sign=false,scoreexact=3}\n')
    >>> # Command \copygroup, with any first argument, and second argument
    >>> # exactly equal to BigGroupe
    >>> TexSurgery(tex).findall(r'\copygroup{category}{@BigGroupe}')
    [('\\copygroup', {'category': 'cat1', '@BigGroupe': 'BigGroupe'}), ('\\copygroup', {'category': 'cat2', '@BigGroupe': 'BigGroupe'})]
    >>> # Command \subsection, with any title as first argument,
    >>> # with \label{seed}
    >>> TexSurgery(tex).find('\\subsection[label="student id"]{title}')
    ('\\subsection', {'title': 'Student identification'})
    >>> # Command \subsection, with any title as first argument,
    >>> # with \label{seed} and in this subsection (which is ended by a \section)
    >>> # there is a run environment
    >>> TexSurgery(tex).find('\\subsection{title}#seed:next run')
    ('\\subsection', ('run', "\nprint('The random seed is ', seed)\n"))

### `insertAfter` and `replace`

`texsurgery` can perform search-and-replace, and search-and-insert-after using the css-style selectors:

.. doctest::

    >>> from texsurgery.texsurgery import TexSurgery
    >>> tex = r'''\begin{choices}
    ... \wrongchoice{$\sage{fd + a}$}
    ... \correctchoice{$\sage{fd}$}
    ... \wrongchoice{$\sage{fd*a}$}
    ... \end{choices}
    ... '''
    >>> ts = TexSurgery(tex)
    >>> ts = ts.replace(r'\correctchoice{choice}', r'\correctchoice{$\sage{f.derivative(x)}$}')
    >>> ts.src
    '\\begin{choices}\n\\wrongchoice{$\\sage{fd + a}$}\n\\correctchoice{$\\sage{f.derivative(x)}$}\n\\wrongchoice{$\\sage{fd*a}$}\n\\end{choices}\n'

### `shuffle`

`texsurgery` can also shuffle some TexElements nested within a parent using the css-style selectors:

.. doctest::

    >>> from texsurgery.texsurgery import TexSurgery
    >>> tex = r'''\begin{choices}
    ... \wrongchoice{$\sage{fd + a}$}
    ... \correctchoice{$\sage{fd}$}
    ... \wrongchoice{$\sage{fd*a}$}
    ... \end{choices}
    ... '''
    >>> ts = TexSurgery(tex)
    >>> ts = ts.shuffle('choices', r'\correctchoice{choice},\wrongchoice{choice}', randomseed=1)
    >>> ts.src
    '\\begin{choices}\n\\correctchoice{$\\sage{fd}$}\n\\wrongchoice{$\\sage{fd*a}$}\n\\wrongchoice{$\\sage{fd + a}$}\n\\end{choices}\n'


## Kernels

TexSurgery can use several jupyter kernels in the same document. In order to do so, you have to declare them with a line like

~~~~~~~~~~latex
\usepackage[sagemath,python3]{texsurgery}
~~~~~~~~~~

The first one is the default. To use another one, pass the option to the corresponding environment. For example
~~~~~~~~~~latex
\begin{run}
1^1
\end{run}
~~~~~~~~~~
will be transformed into
~~~~~~~~~~latex
1
~~~~~~~~~~
but, because of the different way that `sagemath` and `python` handle the `^` operator, this
~~~~~~~~~~latex
\begin{run}[python3]
1^1
\end{run}
~~~~~~~~~~
will be transformed into
~~~~~~~~~~latex
0
~~~~~~~~~~
instead.

This can be useful for example to include graphics generated by different systems in the same document.

## Example

Start with this LaTeX code:

~~~~~~~~~~latex
% Any jupyter kernel is available
\usepackage[sagemath]{texsurgery}

% Compatible with any other LaTeX package
\usepackage[bloc,completemulti]{automultiplechoice}

% Example of user macros
\providecommand{\abs}[1]{\lvert#1\rvert}
\newcommand{\R}{\mathbb{R}}

% TexSurgery can replace some \commands before pdflatex runs
\begin{minipage}{.85\linewidth}
Student: {\bf \name \;  \surname},  \quad ID:  {\bf \id}
\end{minipage}

\begin{question}{derivative-sin}
\qvariant{1} \qtags{derivative}
% TexSurgery will run code in a jupyter kernel
\begin{runsilent}
set_random_seed(\seed)
a = randint(2,10)
f = sin(a*x)
fd = f.derivative(x)
\end{runsilent}
% TexSurgery will eval code in a jupyter kernel
% and replace \eval{expr} with the output from the kernel
% \sage{expr} is just an alias for \eval{latex(expr)}
What is the first derivative of $\sage{f}$?
\begin{choices}
  \correctchoice{$\sage{fd}$}
  \wrongchoice{$\sage{fd*a}$}
  \wrongchoice{$\sage{fd + a}$}
\end{choices}
\begin{explain}
\begin{run}
# TexSurgery will run code in the jupyter kernel
# and replace this environment with the full output
\end{run}
\end{explain}
\end{question}
~~~~~~~~~~

and run this `python` code:
~~~~~~~~~~python
from texsurgery.texsurgery import TexSurgery
student_vars = dict(name='Fulano', surname='de Tal', seed='1', id='314159')
ts = TexSurgery(tex_source).data_surgery(student_vars).code_surgery()
~~~~~~~~~~

in order to transform it into this:

~~~~~~~~~~latex
% Compatible with any other LaTeX package
\usepackage[bloc,completemulti]{automultiplechoice}

% Example of user macros
\providecommand{\abs}[1]{\lvert#1\rvert}
\newcommand{\R}{\mathbb{R}}

\begin{minipage}{.85\linewidth}
Student: {\bf Fulano \;  de Tal},  \quad ID:  {\bf 314159}
\end{minipage}

\begin{question}{derivative-sin}
\qvariant{1} \qtags{derivative}
What is the first derivative of $\sin\left(7 \, x\right)$?
\begin{choices}
  \correctchoice{$7 \, \cos\left(7 \, x\right)$}
  \wrongchoice{$49 \, \cos\left(7 \, x\right)$}
  \wrongchoice{$7 \, \cos\left(7 \, x\right) + 7$}
\end{choices}
\begin{explain}
\begin{run}
# TexSurgery will run code in the jupyter kernel
# and replace this environment with the full output
\end{run}
\end{explain}
\end{question}
~~~~~~~~~~

## Thanks

To all our colleagues that gave feedback to the early versions, specially Fabricio from ETSIN.UPM and Carlos from ETSIAAB.UPM