Package: jupysub
Jupyter Notebook Processing Script DocumentationJupyter Notebook Processing Script DocumentationOverviewGetting StartedCommand-Line InterfaceCommand-line OptionsExample UsageExample of markdown cell before and after replacements:Handling Expected OutputExpected Output in Markdown CellsAuto-linking Functions with Inline Code in Markdown CellsHow to UseExample UsageDetailed ReplacementsMarkdown CellsInline Code ReplacementHint ReplacementExercise Number ReplacementExercise Difficulty ReplacementCode CellsAnswer BlocksExclude BlocksOptional BlocksInline Code ReplacementExampleBefore:After:Hint Replacement (Colored divs)End-of-Script CleanupCustomizationOverviewThis documentation serves as a comprehensive guide for the Jupyter Notebook Processing Script, a Python utility designed to automate various modifications and enhancements on Jupyter Notebooks. Whether you're a student, instructor, or data scientist, this script helps streamline your Jupyter Notebook workflow by handling repetitive tasks such as formatting and running cells, as well as adding useful features like auto-linking to documentation and displaying expected outputs.The script utilizes Python'sargparselibrary for command-line interface, allowing you to specify options such as input and output files, and whether to run the notebook before or after modifications.Here's what the script can do for you:Run Notebook: Before or after making modifications, as specified.Formatting: Automatic conversion of inline code and hints in Markdown cells.Expected Output: Captures and displays the expected output of code cells.Auto-linking: Adds hyperlinks to known functions in Markdown cells.Customizable: Highly configurable through regular expressions and replacement functions.For detailed usage and customization options, please refer to the sections below.Getting StartedCommand-Line InterfaceThe script leverages Python'sargparselibrary to provide a command-line interface. Below are the options that you can specify:Command-line OptionsOptionDescriptionDefault Value-i, --infileSpecifies the path to the input notebook file../main_notebooks/example.ipynb-o, --outfileSpecifies the path to the output notebook file../main_notebooks/example_out.ipynb-rb, --run-beforeRuns the notebook before making any modifications.Not set-ra, --run-afterRuns the notebook after all modifications are made.Not set-sc, --skip-codeSkip replacement of backticks for ``` tagsNot setExample UsageTo illustrate how to use these command-line options, consider the following example:$jupysub-i./main_notebooks/example.ipynb-o./main_notebooks/example_out.ipynb-rb-raRuns the file./main_notebooks/example.ipynbPerforms several substitutions (see more below)Adds an html box under each code cell with the expected cell outputAdds url's to documentations of known functions to markdownAdds difficulty rating to the exercisesAdds question number <#q>Runs the file a second timeSaves the result under./main_notebooks/example_out.ipynbExample of markdown cell before and after replacements:Before:# Sample Jupyter Notebook Markdown CellIn this notebook, we are going to explore data transformation using Python. You can use`pandas`to clean the data and`matplotlib`to plot graphs.>**Note:** Make sure to install all dependencies before running the notebook.>**Warning:** Be careful when manipulating data. Always double-check your code.Here is a Python code cell that includes blocks that will be transformed:

-------------------------#!BEGIN ANSWERx = 10
y = 20#!END ANSWER#!BEGIN EXCLUDEprint("This line will be excluded.")#!END EXCLUDE#!BEGIN OPTIONALprint("This line will be commented out.")#!END OPTIONAL-------------------------After:# Sample Jupyter Notebook Markdown CellIn this notebook, we are going to explore data transformation using Python. You can use <code>pandas</code> to clean the data and <code>matplotlib</code> to plot graphs.

<div class="alert alert-block alert-warning" style="border-radius: 6px">
<b>Note:</b> Make sure to install all dependencies before running the notebook.
</div>

<div class="alert alert-block alert-danger" style="border-radius: 6px">
<b>Warning:</b> Be careful when manipulating data. Always double-check your code.
</div>

Here is a Python code cell that includes blocks that will be transformed:


-------------------------# ADD YOUR CODE HERE# print("This line will be commented out.")-------------------------

Expected output:

-------------------------

This line will be excluded.
This line will be commented out.

-------------------------Handling Expected OutputExpected Output in Markdown CellsMethodology: The script captures the actual output of each code cell upon execution. This can include text or images.Description: Places the captured output in a new Markdown cell below the corresponding code cell. The output is enclosed within a toggleable HTMLdiv, allowing users to choose whether or not they wish to see the expected output.Example:Before:A Python cell that prints "Hello, World!"print("Hello, World!")After:The original Python cell followed by a new Markdown cell with the captured output and toggle feature.print("Hello, World!")Click to toggle expected outputHello, World!Auto-linking Functions with Inline Code in Markdown CellsTheNotebookManipulatorclass allows you to automatically add hyperlinks to functions from libraries like NumPy, Pandas, Matplotlib, SciPy, and the Python Standard Library in markdown cells. This is facilitated through theapply_replacementsmethod and a special functionadd_links_to_code.How to UseBuild the Replacement Dictionary: First, initialize the replacement dictionary by callingNotebookManipulator.build_replacement_dict().replacements=NotebookManipulator.build_replacement_dict()Configure Inline Code Replacements: Add a regular expression pattern for inline code to the "markdown" section of the replacement dictionary. Use theadd_links_to_codefunction for replacements.replacements["markdown"][r'(<code.*?>)(.*?)(</code>)']=add_links_to_codeApply the Replacements: Finally, apply these replacements to the notebook.notebook.apply_replacements(replacements)Example UsageHere's how you could use these steps in a complete example:# Initialize NotebookManipulator and the replacement dictionarynote=NotebookManipulator("your_notebook.ipynb")replacements=NotebookManipulator.build_replacement_dict()# Configure URL replacements for inline codereplacements["markdown"][r'(<code.*?>)(.*?)(</code>)']=add_links_to_code# Apply the replacementsnote.apply_replacements(replacements)# Save the notebooknote.save("your_modified_notebook.ipynb")Sonp.loadtxt()for example would be enclosed in a relatively long html tag that would give link the user to the documentation.Detailed ReplacementsMarkdown CellsInline Code ReplacementRegular Expression Used:inline_code  =r"(?<!)([^]+)(?!)"`Description: Finds inline code snippets encapsulated in backticks and replaces them with HTML<code>tags.Example:Before:`print()`After:<code>print()</code>Hint ReplacementRegular Expression Used:hint = r"^>\s*\**([a-zA-Z]{1,10}:)\**(.*?)$"Description: Transforms Markdown-styled hints into styled HTMLdivblocks.Example:Before:> **Note:** This is a note.After:<div class="alert alert-block alert-warning"><b>Note:</b> This is a note.</div>Exercise Number ReplacementRegular Expression Used: Uses the patternr"^(#.*?)<#(\w+)>"to find sections marked for exercise numbering.Description: Looks for exercise headers containing<#key>and replaces it with an incrementing counter based on the key specified. This is useful for automatic numbering of exercises in a document.Example:Before:## Exercise <#q>: Helo## Exercise <#q>: HelooAfter:## Exercise 1: Helo## Exercise 2: HelooExercise Difficulty ReplacementRegular Expression Used: Uses the patternr"^(#.*?)<d(\d)/(\d)>"to find sections marked for difficulty rating.Description: Replaces<dX/Y>with a visual rating system using emojis.Xis the number of filled emojis to show, andYis the total number of emojis. Emojis with reduced opacity represent the remaining difficulty points.Example:Before:## Question <#q>: Heat transfer <d3/5>## Question <#q>: Another Heat transfer <d2/5>After:## Question 1: Heat transfer <span style="font-size: 10pt;"><span style="opacity: 1;">🧠🧠🧠</span><span style="opacity: 0.1;">🧠🧠</span></span>## Question 2: Another Heat transfer <span style="font-size: 10pt;"><span style="opacity: 1;">🧠🧠</span><span style="opacity: 0.1;">🧠🧠🧠</span></span>Code CellsAnswer BlocksRegular Expression Used: Uses a generic pattern to find blocks taggedANSWER.Description: Replaces the entire code block tagged withANSWERwith a placeholder for students or users to fill in.Example:Before:#!BEGIN ANSWERx=10y=20#!END ANSWERAfter:# ADD YOUR CODE HEREExclude BlocksRegular Expression Used: Uses a generic pattern to find blocks taggedEXCLUDE.Description: Deletes the entire code block tagged withEXCLUDE.Example:Before:#!BEGIN EXCLUDEprint("This should be excluded.")#!END EXCLUDEAfter:</code></pre></li></ul></li></ul><h4>OptionalBlocks</h4><ul><li><p><strong>RegularExpressionUsed</strong>:Usesagenericpatterntofindblockstagged<code>OPTIONAL</code>.</p></li><li><p><strong>Description</strong>:Commentsoutthecodeinsideblockstagged<code>OPTIONAL</code>.</p></li><li><p><strong>Example</strong>:</p><ul><li><strong>Before</strong>:<prelang="python"><code>#!BEGIN OPTIONALprint("This is optional.")#!END OPTIONALAfter:# print("This is optional.")Inline Code ReplacementThis operation targets inline code elements in Markdown cells. The aim is to ensure that the inline code renders correctly when the notebook is converted to HTML or other formats.ExampleBefore:Here, we are talking about `x` and `y`.After:Here, we are talking about `<code>x</code>` and `<code>y</code>`.Hint Replacement (Colored divs)Regular Expression Used:hint = r"^>\s*\**([a-zA-Z]{1,10}:)\**(.*?)$"Description: Transforms Markdown-styled hints into styled HTMLdivblocks. Thedivclass changes based on the type of hint, meaning the hint box will be colored differently for different types of hints.Examples:Type: WarningBefore:> **Warning:** Be careful with this step.After:<divclass="alert alert-block alert-danger"style="border-radius: 6px"><b>Warning:</b>Be careful with this step.</div>Type: NoteBefore:> **Note:** This is an important point.After:<divclass="alert alert-block alert-warning"style="border-radius: 6px"><b>Note:</b>This is an important point.</div>Type: InfoBefore:> **Info:** Additional details here.After:<divclass="alert alert-block alert-info"style="border-radius: 6px"><b>Info:</b>Additional details here.</div>End-of-Script CleanupRegular Expression Used:end_of_script = r"([\n \t]*)\Z"Description: Removes trailing white spaces and new lines at the end of each cell.Example:Before:print("Hello World")After:print("Hello World")CustomizationThe script is designed to be highly customizable. Feel free to change the regular expressions, the replacement functions, or even the order of the operations to suit your specific needs.