Package: cauldron-notebook
Interactive computing for complex data processing, modeling and analysis
in Python 3.Major New ReleaseHeads up as Cauldron v1.0.0 has just been released with a new web-based
user interface. More detailed documentation on how the new UI works can
be found athttp://www.unnotebook.comThe rest of this documentation pertains to the Cauldron command line
interface. For more general information about Cauldron, including how
to use the UI, please visit:http://www.unnotebook.comInstallationGetting StartedExample ProjectsTutorial: First ProjectDockerInstallationThe latest release of Cauldron is available from both PyPi:$ pip install cauldron-notebookand Anaconda:$ conda install -c sernst cauldronIf you want to use the latest developments, you can install directly from the Github
page instead of from PyPi:$ pip install git+https://github.com/sernst/cauldron.gitYou can also install in development mode if you want to manage updates using git
instead of pip. To install in that way, clone a local copy of this repository
to your local machine and, inside a terminal,cdinto your local copy
directory and run the command:$ python3 setup.py developOr in an Anaconda installation using its develop command:$ conda develop .which must be executed in the root project directory of your local copy of
Cauldron.Getting StartedCauldron can be used as either through its Command Line Interface (CLI) or with
the Cauldron web-based UI. For more information about the UI visithttp://www.unnotebook.comfor examples and documentation. The rest of this
README describes using Cauldron directly from the command line.Cauldron is a shell-based program you start from a terminal. For installations
that support python script installation you can start Cauldron
once the installation is complete with thecauldroncommand:$ cauldronor on Windows using thecauldron.execommand:% cauldron.exeFor installations where the installation of scripts was not permitted, you can
start Cauldron from within a Python shell. To do this import cauldron and
run thecauldron.run_shell()function as follows:>>> import cauldron
>>> cauldron.run_shell()Once started, the Cauldron shell provides all of the functionality you need to
manage your analysis projects through a collection of commands. To see a list
of available commands and their basic descriptions use the?orhelpcommand on the Cauldron prompt:<>: ?or:<>: helpFor more detailed information on a specific command use thehelpcommand
along with the name of the command you wish to learn more about. For example,
to get help on theopencommand, you would enter:<>: help openon the Cauldron prompt.Example ProjectsCauldron comes bundled with a few example projects for demonstration purposes.
To open one of these projects, use the command:<>: open @examples:[EXAMPLE_PROJECT_NAME]where[EXAMPLE_PROJECT_NAME]is the name of an existing example project.
The@examples:prefix is an alias in Cauldron that resolves to the path
where the example files are stored. You can also create your own aliases,
which will be explained in detail later.Like all commands in Cauldron, the open command supports tab auto-completion.
If you enter the beginning of the command above:<>: open @examples:and hit the tab key with the cursor at the end of the line, Cauldron will give
you a list of the example project subdirectories.A good example to start would be Cauldron’shello_cauldron:<>: open @examples:hello_cauldron/Once this command is run, the hello_cauldron project will be opened and readied
for you to run. The Cauldron shell prompt updates to reflect the open project.
Instead of<>:, which signifies no open project, the prompt should now be<hello_cauldron>:.If you now enter theruncommand without any arguments, all steps (cells)
in the project will run:<hello_cauldron>: runOnce complete, you can view the current state of the notebook display with the
show command:<hello_cauldron>: showwhich opens the current project display file in your default browser. When you
are finished working on a project, you use the close to command:<hello_cauldron>: closeThis empties all of the information Cauldron has been storing for your project
in memory, and takes you back to the initial command prompt where you started:<>:Tutorial: First ProjectThis tutorial walks through creating your first project. It mirrors the@example:hello_cauldronproject that comes installed with Cauldron.Create New ProjectTo create your first project run the Cauldron shell command:<>: create hello_cauldron @home:For more details about the create command, use the Cauldron shell command:<>: help createThe create command takes two arguments:The name of your new project (hello_cauldronin the example above)The absolute path to the directory where the project will be saved. In the
example above, the@home:argument is a shortcut to Cauldron’s default
home directory, which is ~/cauldron/.When the example create command above is executed, ahello_cauldronproject
will be created in the directory~/cauldron/hello_cauldron/, with the
scaffolding for the project already written. The create command also
immediately opens the new project in the shell.Add First Code StepNow that the project has been created, you need to add some code to it. To
do that, use thesteps addcommand:<hello_cauldron>: steps add create_data.pyThis will create a new step calledS01-create_data.pyin your project
directory and add it to the Cauldron project. Notice that the name you gave
the step and the one actual step name are different. There’s anS01-prefix
added to the file. This prefix is added automatically by Cauldron to help you
organize your files. You can disable this feature when you create a project if
you really want to manage the names all yourself, but we’ll get into that in
an advanced tutorial.The step file you created is ready to be modified. Open theS01-create_data.pystep file in your choice of Python code editor. You’ll
find the file in the project directory, which is~/cauldron/hello_cauldron/.
Add the following code to theS01-create_data.pyfile:importnumpyasnpimportpandasaspdimportcauldronascddf=pd.DataFrame(np.random.randn(10,5),columns=['a','b','c','d','e'])cd.display.header('Random Data Frame:')cd.display.table(df)cd.shared.df=dfOnce you’ve saved that code to theS01-create_data.pyfile, you can run your
project using theruncommand:<hello_cauldron>: runThen use theshowcommand to see the results:<hello_cauldron>: showThe project display file will open in your default browser.Add Another StepNow we’ll add another code step to plot each column in our DataFrame. Once
again use the steps command:<hello_cauldron>: steps add plot_data.pyOpen theS02-plot_data.pystep file and add the following code:importmatplotlib.pyplotaspltimportcauldronascddf=cd.shared.dfforcolumn_nameindf.columns:plt.plot(df[column_name])plt.title('Random Plot')plt.xlabel('Indexes')plt.ylabel('Values')cd.display.pyplot()We used matplotlib for this tutorial, but Cauldron also supports Seaborn,
Bokeh, Plotly or any other Python plotting library that can produce an HTML
output. There are Cauldron example projects showing how to plot using each of
these libraries.Now run the project again:<hello_cauldron>: runYou’ll notice that the shell output looks like:=== RUNNING ===
[S01-create_data.py]: Nothing to update
[S02-plot_data.py]: UpdatedTheS01-create_data.pystep was not run because it hasn’t been modified since
the last time you executed theruncommand. Just like other notebooks, the
results of running a step (cell) persist until you close the project and do not
need to be updated each time. Cauldron watches for changes to your files and
only updates steps if the files have been modified, or an early step was
modified that may affect their output.Now you can view the updated project display simply by refreshing your browser.
However, if you already closed the project display browser window, you can show
it again at any time with theshowcommand.And that’s that. You’ve successfully created your first Cauldron project. You
can close your project with theclosecommand:<hello_cauldron>: closeOr, if you want to exit the Cauldron shell at any time, use theexitcommand:<>: exitSee Cauldron’s documentation athttp://www.unnotebook.com/docs/for more
information.DockerCauldron supports running in docker containers for both local and remote
workflows. There are 3 officially supported docker containers available at:https://hub.docker.com/r/swernst/cauldron/tags/They are:standard: Includes the Python 3.6+ distributions in an Ubuntu environment.conda: Includes the full Anaconda distribution of Python 3.6+ built upon Anaconda’s official docker image.miniconda: Includes the slimmed-down mini Anaconda distribution of Python 3.6+ built upon Anaconda’s official docker image.In all three cases, Cauldron is pre-installed with dependencies and the default
command for each container is to start the Cauldron kernel on the exposed port
5010. One of these containers can be pulled using the docker pull command:$ docker pull swernst/cauldron:current-standardIf you do not specify a specific tag, the latest standard image will be used.
Once the image has been pulled, you can start a Cauldron kernel:$ docker run -d --rm -p 5010:5010 swernst/cauldron:current-standardAfter the container starts, you can access the kernel through the exposed 5010
port. If you are using the Cauldron UI, you can connect to this
container locally by specifying the local kernel URL,http://127.0.0.1:5010,
as connection argument when starting the ui, .The Cauldron command shell also allows you drive the kernel by connecting to it
from a locally running Cauldron shell. To do this, you use theconnectcommand:<>: connect http://127.0.0.1:5010Once connected, all shell commands you issue, e.g. opening a project, will be
relayed to the kernel. All project files will be synchronized between the
local environment and the kernel’s environment. This means you can interact
with a local project exactly like you normally would, but all of the execution
will happen in the kernel’s environment, not your local one.Windows DevelopmentIf you are developing and testing on Windows, you will need to installpyreadline3for the test suite to run as thereadlinestandard library package is not available
on Windows.