Package: davos
davosSomeone once told me that the night is dark and full of terrors. And tonight I am no knight. Tonight I am Davos the
smuggler again. Would that you were an onion.—Ser Davos SeaworthA Clash of KingsbyGeorge R. R. MartinIntroductionThedavoslibrary provides Python with an additional keyword:smuggle.Thesmugglestatementworks just like the built-inimportstatement, with two major differences:You cansmugglea packagewithout installing it firstYou cansmuggleaspecific versionof a packageTaken together, these two enhancements toimportprovide a powerful system for developing and sharing reproducible code that works across different users and environments.Table of contentsTable of contentsIntroduction (↑)Why would I want an alternative toimport?Why not use virtual environments, containers, and/or virtual machines instead?InstallationLatest Stable PyPI ReleaseLatest GitHub UpdateInstalling in ColaboratoryOverviewSmuggling Missing PackagesSmuggling Specific Package VersionsUse CasesSimplify sharing reproducible code & Python environmentsGuarantee your code always uses the latest version, release, or revisionCompare behavior across package versionsUsageThesmuggleStatementSyntaxRulesThe Onion CommentSyntaxRulesThedavosConfigReferenceTop-level FunctionsHow It Works: ThedavosParserAdditional NotesWhy would I want an alternative toimport?In many cases,smuggleandimportdo the same thing—if you're
running code in the same environment you developed it in. But what if you want
to share aJupyter notebookcontaining your code with
someone else? If the user (i.e., the "someone else" in this example) doesn't
have all of the packages your notebook imports, Python will raise an exception
and the code won't run. It's not a huge deal, of course, but it's inconvenient
(e.g., the user might need topip-install the missing packages, restart their
kernel, re-run the code up to the point it crashed, etc.—possibly going
through this cycle multiple times until the thing finally runs).A second (and more subtle) issue arises when the developer (i.e., the person
whowrotethe code) used or assumed different versions of the imported
packages than what the user has installed in their environment. So maybe the
original author was developing and testing their code usingpandas1.3.5, but
the user hasn't upgraded theirpandasinstallation since 0.25.0. Python will
happily "import pandas" in both cases, but any changes across those versions
might change what the developer's code actually does in the user's (different)
environment—or cause it to fail altogether.The problemdavostries to solve is similar to the idea motivating virtual
environments, containers, and virtual machines: we want a way of replicating
the original developer's environment on the user's machine, to a sufficiently
good approximation that we can be "reasonably confident" that the code will
continue to behave as expected.When yousmugglepackages instead of importing them, it guarantees (for
whatever environment the code is running in) that the packages are importable,
even if they hadn't been installed previously. Under the hood,davosfigures
out whether the package is available, and if not, it usespipto download and
install anything that's missing (including missing dependencies). From that
point, after having automatically handled those sorts of dependency issues,smugglebehaves just likeimport.The second powerful feature ofdavoscomes from another construct, called
"onion comments." These are like standard Python
comments, but they appear on the same line(s) assmugglestatements, and they
are formatted in a particular way. Onion comments provide a way of precisely
controlling how, when, and where packages are installed, how (or if) the system
checks for existing installations, and so on. A key feature is the ability to
specify exactly which version(s) of each package are imported into the current
workspace. When used in this way,davosenables authors to guarantee that the
same versions of the packages they developed their code with will also be
imported into the user's workspace at the appropriate times.Why not use virtual environments, containers, and/or virtual machines instead?Psst-- we'll let you in on a little secret: importingdavosautomaticallycreates a virtual environment for your notebook. However, whereas setting up a
virtual environment is usually left to the user,davoshandles the pesky
details for you, without you needing to think about them. Any packages yousmuggleviadavosthat aren't available in the notebook's original runtime
environment are installed into a new virtual environment. This ensures thatdavoswill not change the runtime environment (e.g., by installing new
packages, changing existing package versions, etc.).By default, each notebook's virtual environment is stored in a hidden ".davos"
folder inside the current user's home directory. The default environment name
is computed to uniquely identify each notebook, according to its filename and
path. However, a notebook's virtual environment may be customized by settingdavos.projectto any string that can be used as a valid folder name in the
user's operating system. This is useful for multi-notebook projects that share
dependencies (without needing to duplicate each package installation for each
notebook).If you prefer, you can also disabledavos's virtual environment
infrastructure by settingdavos.projecttoNone. Doing so will cause any
packages installed bydavosto affect the notebook's runtime environment.
This is generally not recommended, as it can lead to unintended consequences
for other code that shares the runtime environment. That said,davosalso
works great when used inside of (standard) virtual environments, containers,
and virtual machines.There are a few additional specific advantages todavosthat go beyond more
typical virtual environments, containers, and/or virtual machines. The main
advantage is thatdavosis very lightweight: importingdavosinto a
notebook-based environment unlocks all of its functionality without needed to
install, set up, and learn how to use additional stuff. There is none of the
typical overhead of setting up a new virtual environment (or container, virtual
machine, etc.), installing third-party tools, writing and sharing configuration
files, and so on. All of your codeand its dependenciesmay be contained in a
single notebook file.Okay... so how do I use this thing?To turn a standardJupyter (IPython) notebook, including aGoogle Colaboratory notebook, into adavos-enhanced notebook, just add two lines to the first cell:%pipinstalldavosimportdavosThis will enable thesmugglekeyword in your notebook environment.  Then you can do things like:# pip-install numpy v1.23.1, if neededsmugglenumpyasnp# pip: numpy==1.23.1# the smuggled package is fully imported and usablearr=np.arange(15).reshape(3,5)# and the onion comment guarantees the desired version!assertnp.__version__=='1.23.1'Interested? Curious? Intrigued? Check out the table of contents for more
details! You may also want to check out ourpaperfor more
formal descriptions and explanations.InstallationLatest Stable PyPI ReleasepipinstalldavosLatest GitHub Updatepipinstallgit+https://github.com/ContextLab/davos.gitInstalling in ColaboratoryTo installdavosinGoogle Colab, add a new cell to the top of your notebook with an
percentage sign (%) followed by one of the commands above (e.g.,%pip install davos). You'll likely also want toimport davos,
which enables thesmugglesyntax.  Run the cell to installdavoson the runtime virtual machine.Note: restarting the Colab runtime does not affect installed packages. However, if the runtime is "factory reset"
or disconnected due to reaching its idle timeout limit, you'll need to rerun the cell to reinstalldavoson the fresh
VM instance.OverviewThe primary way to usedavosis viathesmugglestatement, which is made available
simply by runningimport davos. Likethe built-inimportstatement, thesmugglestatement is used to
load packages, modules, and other objects into the current namespace. The main difference between the two is in how
they handle missing packages and specific package versions.Smuggling Missing Packagesimportrequires that packages be installedbeforethe start of the interpreter session. Trying toimporta package
that can't be found locally will throw aModuleNotFoundError, and you'll have to
install the package from the command line, restart the Python interpreter to make the new package importable, and rerun
your code in full in order to use it.Thesmugglestatement, however, can handle missing packages on the fly. If yousmugglea package that isn't
installed locally,davoswill install it for you, make its contents available to Python'simport machinery, and load it into the namespace for immediate use.
You can controlhowdavosinstalls missing packages by adding a special type of inline comment called an"onion" commentnext to asmugglestatement.Smuggling Specific Package VersionsOne simple but powerful use foronion commentsis makingsmugglestatementsversion-sensitive.Python doesn't provide a native, viable way to ensure a third-party package imported at runtime matches a specific
version or satisfies a particularversion constraint.
Many packages expose their version info via a top-level__version__attribute (seePEP 396), and certain tools (such as the standard library'simportlib.metadataandsetuptools'spkg_resources) attempt to parse version info from
installed distributions. However, using these to constrain imported package would require writing extra code to compare
version strings andstillmanually installing the desired version and restarting the interpreter any time an
invalid version is caught.Additionally, for packages installed through a version control system (e.g.,git), this would be
insensitive to differences between revisions (e.g., commits) within the same semantic version.davossolves these issuesby allowing you to specify a specific version or set of acceptable versions for each
smuggled package. To do this, simply provide aversion specifierin anonion commentnext to thesmugglestatement:smugglenumpyasnp# pip: numpy==1.23.1frompandassmuggleDataFrame# pip: pandas>=1.0,<2.0In this example, the first line will loadnumpyinto the local namespace under the alias "np",
just as "import numpy as np" would. First,davoswill check whethernumpyis installed locally, and if so, whether
the installed versionexactlymatches1.23.1. Ifnumpyis not installed, or the installed version is anything
other than1.23.1,davoswill use the specifiedinstaller program,pip, to
installnumpy==1.23.1before loading the package.Similarly, the second line will load the "DataFrame" object from thepandaslibrary,
analogously to "from pandas import DataFrame". A localpandasversion of1.2.1would be used, but a local version
of2.1.1would causedavosto replace it with a validpandasversion, as if you had manually runpip install pandas>=1.0,<2.0.In both cases, the imported versions will fit the constraints specified in theironion comments,
and the next timenumpyorpandasis smuggled with the same constraints, valid local installations will be found.You can also force the state of a smuggled packages to match a specific VCS ref (branch, revision, tag, release, etc.).
For example:smugglehypertoolsashyp# pip: git+https://github.com/ContextLab/hypertools.git@98a3d80will loadhypertools(aliased as "hyp"), as the package existedon GitHub, at commit98a3d80. The general format for VCS references inonion commentsfollows that of thepip-installcommand. See thenotes on smuggling from VCSbelow for additional info.And witha few exceptions, smuggling a specific package version will workeven if the package
has already been imported!Note:davosv0.2.x supportsIPythonenvironments  (e.g.,JupyterandColaboratorynotebooks) only. v0.3.x will add
support for "regular" (i.e., non-interactive) Python scripts.Use CasesSimplify sharing reproducible code & Python environmentsDifferent versions of the same package can often behave quite differently—bugs are introduced and fixed, features
are implemented and removed, support for Python versions is added and dropped, etc. Because of this, Python code that is
meant to bereproducible(e.g., tutorials, demos, data analyses) is commonly shared alongside a set of fixed versions
for each package used. And since there is no Python-native way to specify package versions at runtime (seeabove), this typically takes the form of a pre-configured development
environment the end user must build themselves (e.g., aDockercontainer orcondaenvironment), which can be cumbersome, slow to set up, resource-intensive, and
confusing for newer users, as well as require shipping both additional specification filesandsetup instructions
along with your code. And even then, a well-intentioned user may alter the environment in a way that affects your
carefully curated set of pinned packages (such as installing additional packages that trigger dependency updates).Instead,davosallows you to share code with one simple instruction:justpip install davos!Replace yourimportstatements withsmugglestatements, pin package versions in onion comments, and letdavostake care of the rest.
Beyond its simplicity, this approach ensures your predetermined package versions are in place every time your code is
run.Guarantee your code always uses the latest version, release, or revisionIf you want to make sure you're always using the most recent release of a certain package,davosmakes doing so easy:smugglemypkg# pip: mypkg --upgradeOr if you have an automation designed to test your most recent commit on GitHub:smugglemypkg# pip: git+https://username/reponame.gitCompare behavior across package versionsThe ability tosmugglea specific package version even after a different version has been imported makesdavosa
useful tool for comparing behavior across multiple versions of the same package, within the same interpreter session:deftest_my_func_unchanged():"""Regression test for `mypkg.my_func()`"""data=list(range(10))smugglemypkg# pip: mypkg==0.1result1=mypkg.my_func(data)smugglemypkg# pip: mypkg==0.2result2=mypkg.my_func(data)smugglemypkg# pip: git+https://github.com/MyOrg/mypkg.gitresult3=mypkg.my_func(data)assertresult1==result2==result3UsageThesmuggleStatementSyntaxThesmugglestatement is meant to be used in place ofthe built-inimportstatementand sharesits full syntactic definition:smuggle_stmt::="smuggle"module["as"identifier](","module["as"identifier])*|"from"relative_module"smuggle"identifier["as"identifier](","identifier["as"identifier])*|"from"relative_module"smuggle""("identifier["as"identifier](","identifier["as"identifier])*[","]")"|"from"module"smuggle""*"module::=(identifier".")*identifierrelative_module::="."*module|"."+NB: uses the modified BNF grammar notation described inThe Python Language Reference,here; seeherefor the lexical definition
    ofidentifierIn simpler terms,any valid syntax forimportis also valid forsmuggle.RulesLikeimportstatements,smugglestatements are whitespace-insensitive, unless a lack of whitespace between two
tokens would cause them to be interpreted as a different token:fromos.pathsmuggledirname,joinasopj# validfromos.pathsmuggledirname,joinasopj# also validfromos.pathsmuggledirname,joinasopj# invalid ("asopj" != "as opj")Any context that would cause animportstatementnotto be executed will have the same effect on asmugglestatement:# smuggle matplotlib.pyplot as plt           # not executedprint('smuggle matplotlib.pyplot as plt')# not executedfoo="""smuggle matplotlib.pyplot as plt"""# not executedBecause thedavosparser is less complex than the full Python parser, there are two fairly non-disruptive edge
cases where animportstatement would be syntactically valid but asmugglestatement would not:Theexecfunctionexec('from pathlib import Path')# executedexec('from pathlib smuggle Path')# raises SyntaxErrorA one-linecompound statementclause:ifTrue:importrandom# executedifTrue:smugglerandom# raises SyntaxErrorwhileTrue:importmath;break# executedwhileTrue:smugglemath;break# raises SyntaxErrorfor_inrange(1):importjson# executedfor_inrange(1):smugglejson# raises SyntaxError# etc...InIPythonenvironments (e.g.,Jupyter&Colaboratorynotebooks)smugglestatements always load names into the global
namespace:# example.ipynbimportdavosdefimport_example():importdatetimedefsmuggle_example():smuggledatetimeimport_example()type(datetime)# raises NameErrorsmuggle_example()type(datetime)# returnsThe Onion CommentAnonion commentis a special type of inline comment placed on a line containing asmugglestatement. Onion comments
can be used to control howdavos:determines whether the smuggled package should be installedinstalls the smuggled package, if necessaryOnion comments are also useful when smuggling a package whosedistribution name(i.e., the name
used when installing it) is different from itstop-level module name(i.e., the name used when importing it). Take for
example:fromsklearn.decompositionsmugglepca# pip: scikit-learnThe onion comment here (# pip: scikit-learn) tellsdavosthat if "sklearn" does not exist
locally, the "scikit-learn" package should be installed.SyntaxOnion comments follow a simple but specific syntax, inspired in part by thetype comment syntaxintroduced inPEP 484. The following is a loose (pseudo-)syntactic definition for an onion
comment:onion_comment::="#"installer":"install_opt*pkg_spec install_opt*installer::=("pip"|"conda")pkg_spec::=identifier[version_spec]NB: uses the modified BNF grammar notation described inThe Python Language Reference,here; seeherefor the lexical definition
    ofidentifierwhereinstalleris the program used to install the package;install_optis any option accepted by the installer's
"install" command; andversion_specmay be aversion specifierdefined byPEP 440followed by aversion string, or an alternative syntax valid
for the giveninstallerprogram. For example,pipuses specific syntaxes forlocal,editable, andVCS-basedinstallation.Less formally,an onion comment simply consists of two parts, separated by a colon:the name of the installer program (e.g.,pip)arguments passed to the program's "install" commandThus, you can essentially think of writing an onion comment as taking the full shell command you would run to install
the package, and replacing "install" with ":". For instance, the command:pipinstall-I--no-cache-dirnumpy==1.23.1-vvv--timeout30is easily translated into an onion comment as:smugglenumpy# pip: -I --no-cache-dir numpy==1.23.1 -vvv --timeout 30In practice, onion comments are identified as matches for theregular expression:#+ *(?:pip|conda) *: *[^#\n ].+?(?= +#| *\n| *$)NB: support for installingsmuggled packages viacondawill be added in v0.2. For v0.1,
    "pip" should be used exclusively.Note: support for installing smuggled packages via thecondapackage manager
will be added in v0.2. For v0.1, onion comments should always specify "pip" as theinstallerprogram.RulesAn onion comment must be placed on the same line as asmugglestatement; otherwise, it is not parsed:# assuming the dateutil package is not installed...# pip: python-dateutil                       # <-- has no effectsmuggledateutil# raises InstallerError (no "dateutil" package exists)smuggledateutil# raises InstallerError (no "dateutil" package exists)# pip: python-dateutil                       # <-- has no effectsmuggledateutil# pip: python-dateutil   # installs "python-dateutil" package, if necessaryAn onion comment may be followed by unrelated inline comments as long as they are separated by at least one space:smuggletqdm# pip: tqdm>=4.46,<4.60 # this comment is ignoredsmuggletqdm# pip: tqdm>=4.46,<4.60            # so is this onesmuggletqdm# pip: tqdm>=4.46,<4.60# but this comment raises OnionArgumentErrorAn onion comment must be the first inline comment immediately following asmugglestatement; otherwise, it is not
parsed:smugglenumpy# pip: numpy!=1.19.1        # <-- guarantees smuggled version is *not* v1.19.1smugglenumpy# has no effect -->         # pip: numpy==1.19.1This also allows you to easily "comment out" onion comments:smugglenumpy## pip: numpy!=1.19.1       # <-- has no effectOnion comments are generally whitespace-insensitive, but installer arguments must be separated by at least one space:fromumapsmuggleUMAP# pip: umap-learn --user -v --no-clean     # validfromumapsmuggleUMAP#pip:umap-learn --user     -v    --no-clean    # also validfromumapsmuggleUMAP# pip: umap-learn --user-v--no-clean       # raises OnionArgumentErrorOnion comments have no effect on standard library modules:smugglethreading# pip: threading==9999  # <-- has no effectWhen smuggling multiple packages with asinglesmugglestatement, an onion comment may be used to refer to thefirstpackage listed:smugglenilearn,nibabel,nltools# pip: nilearn==0.7.1If multipleseparatesmugglestatements are placed on a single line, an onion comment may be used to refer to thelaststatement:smugglegensim;smugglespacy;smugglenltk# pip: nltk~=3.5 --preFor multilinesmugglestatements, an onion comment may be placed on the first line:fromscipy.interpolatesmuggle(# pip: scipy==1.6.3interp1d,interpnasinterp_ndgrid,LinearNDInterpolator,NearestNDInterpolator,)... or on the last line:fromscipy.interpolatesmuggle(interp1d,# this comment has no effectinterpnasinterp_ndgrid,LinearNDInterpolator,NearestNDInterpolator)# pip: scipy==1.6.3... though the first line takes priority:fromscipy.interpolatesmuggle(# pip: scipy==1.6.3    # <-- this version is installedinterp1d,interpnasinterp_ndgrid,LinearNDInterpolator,NearestNDInterpolator,)# pip: scipy==1.6.2                                   # <-- this comment is ignored... and all commentsnoton the first or last line are ignored:fromscipy.interpolatesmuggle(interp1d,# pip: scipy==1.6.3    # <-- ignoredinterpnasinterp_ndgrid,LinearNDInterpolator,# unrelated comment    # <-- ignoredNearestNDInterpolator)# pip: scipy==1.6.2    # <-- parsedThe onion comment is intended to describe how a specific smuggled package should be installed if it is not found
locally, in order to make it available for immediate use. Therefore, installer options that either (A) install
packages other than the smuggled package and its dependencies (e.g., from a specification file), or (B) cause the
smuggled package not to be installed, are disallowed. The options listed below will raise anOnionArgumentError:-h,--help-r,--requirement-V,--versionThedavosConfigThedavosconfig object stores options and data that affect howdavosbehaves. After importingdavos, the config
instance (a singleton) for the current session is available asdavos.config, and its various fields are accessible as
attributes. The config object exposes a mixture of writable and read-only fields. Mostdavos.configattributes can be
assigned values to control aspects ofdavosbehavior, while others are available for inspection but are set and used
internally. Additionally, certain config fields may be writable in some situations but not others (e.g. only if the
importing environment supports a particular feature). Once set,davosconfig options last for the lifetime of the
interpreter (unless updated); however, they donotpersist across interpreter sessions. A full list ofdavosconfig
fields is availablebelow:ReferenceFieldDescriptionTypeDefaultWritable?activeWhether or not thedavosparser should be run on subsequent input (cells, in Jupyter/Colab notebooks). Setting toTrueactivates thedavosparser, enables thesmugglekeyword, and injects thesmuggle()function into the user namespace. Setting toFalsedeactivates thedavosparser, disables thesmugglekeyword, and removes "smuggle" from the user namespace (if it holds a reference to thesmuggle()function). SeeHow it Worksfor more info.boolTrue✅auto_rerunIfTrue, when smuggling a previously-imported package that cannot be reloaded (seeSmuggling packages with C-extensions),davoswill automatically restart the interpreter and rerun all code up to (and including) the currentsmugglestatement. Otherwise, issues a warning and prompts the user with buttons to either restart/rerun or continue running.boolFalse✅ (Jupyter notebooks only)confirm_installWhether or notdavosshould require user confirmation ([y/n]input) before installing a smuggled packageboolFalse✅environmentA label describing the environment into whichdavoswas running. Checked internally to determine which interchangeable implementation functions are used, whether certain config fields are writable, and various other behaviorsLiteral['Python', 'IPython<7.0', 'IPython>=7.0', 'Colaboratory']N/A❌ipython_shellThe global IPython interactive shell instanceIPython.core.interactiveshell.InteractiveShellN/A❌noninteractiveSet toTrueto rundavosin non-interactive mode (all user input and confirmation will be disabled).NB:1. Setting toTruedisablesconfirm_installif previously enabled2. Ifauto_rerunisFalsein non-interactive mode,davoswill throw an error if a smuggled package cannot be reloadedboolFalse✅ (Jupyter notebooks only)pip_executableThe path to thepipexecutable used to install smuggled packages. Must be a path (strorpathlib.Path) to a real file. Default is programmatically determined from Python environment; falls back tosys.executable -m pipif executable can't be foundstrpipexe path orsys.executable -m pip✅smuggledA cache of packages smuggled during the current interpreter session. Formatted as adictwhose keys are package names and values are the (.split()and';'.join()ed) onion comments. Implemented this way so that any non-whitespace change to installer arguments  re-installationdict[str, str]{}❌suppress_stdoutIfTrue, suppress all unnecessary output issued by bothdavosand the installer program. Useful when smuggling packages that need to install many dependencies and therefore generate extensive output. If the installer program throws an error while output is suppressed, both stdout & stderr will be shown with the tracebackboolFalse✅Top-level Functionsdavosalso provides a few convenience for reading/setting config values:davos.activate()Activate thedavosparser, enable thesmugglekeyword, and inject thesmuggle()function into the namespace.
Equivalent to settingdavos.config.active = True. SeeHow it Worksfor more info.davos.deactivate()Deactivate thedavosparser, disable thesmugglekeyword, and remove the namesmugglefrom the namespace if (and
only if) it refers to thesmuggle()function. Ifsmugglehas been overwritten with a different value, the variable
will not be deleted. Equivalent to settingdavos.config.active = False. SeeHow it Worksfor moreinfo.davos.is_active()Return the current value ofdavos.config.active.davos.configure(**kwargs)Set multipledavos.configfields at once by passing values as keyword arguments, e.g.:importdavosdavos.configure(active=False,noninteractive=True,pip_executable='/usr/bin/pip3')is equivalent to:importdavosdavos.active=Falsedavos.noninteractive=Truedavos.pip_executable='/usr/bin/pip3'How It Works: ThedavosParserFunctionally, importingdavosappears to enable a new Python keyword, "smuggle". However,davosdoesn't actually
modify the rules orreserved keywordsused by
Python's parser and lexical analyzer in order to do so—in fact, modifying the Python grammar is not possible at
runtime and would require rebuilding the interpreter. Instead, inIPythonenivonments likeJupyterandColaboratorynotebooks,davosimplements thesmugglekeyword via a combination of namespace injections and its own (far simpler) custom parser.Thesmugglekeyword can be enabled and disabled at will by "activating" and "deactivating"davos(see thedavosConfig ReferenceandTop-level Functions, above). Whendavosis
imported, it is automatically activated by default. Activatingdavostriggers two things:Thesmuggle()functionis injected into theIPythonuser namespaceThedavosparseris registered as acustom input transformerIPython preprocesses all executed code as plain text before it is sent to the Python parser in order to handle
special constructs like%magicand!shellcommands.davoshooks into this process to transformsmugglestatements into syntactically valid Python code. Thedavosparser usesthis regular expressionto match each
line of code containing asmugglestatement (and, optionally, an onion comment), extracts information from its text,
and replaces it with an analogous call to thesmuggle()function. Thus, even though the code visible to the user may
containsmugglestatements, e.g.:smugglenumpyasnp# pip: numpy>1.16,<=1.24 -vvthe code that is actually executed by the Python interpreter will not:smuggle(name="numpy",as_="np",installer="pip",args_str="""numpy>1.16,<=1.24 -vv""",installer_kwargs={'editable':False,'spec':'numpy>1.16,<=1.24','verbosity':2})Thedavosparser can be deactivated at any time, and doing so triggers the opposite actions of activating it:The name "smuggle" is deleted from theIPythonuser namespace,unless it has been overwritten and no longer
refers to thesmuggle()functionThedavosparser input transformer is deregistered.Note: in Jupyter and Colaboratory notebooks, IPython parses and transforms all text in a cell before sending it
to the kernel for execution. This means that importing or activatingdavoswill not make thesmugglestatement
available until thenextcell, because all lines in the current cell were transformed before thedavosparser was
registered. However,deactivatingdavosdisables thesmugglestatement immediately—although thedavosparser will have already replaced allsmugglestatements withsmuggle()function calls, removing the function from
the namespace causes them to throwNameError.Additional NotesReimplementing installer programs' CLI parsersThedavosparser extracts info from onion comments by passing them to a (slightly modified) reimplementation of
their specified installer program's CLI parser. This is somewhat redundant, since the arguments will eventually be
re-parsed by theactualinstaller program if the package needs to be installed. However, it affords a number of
advantages, such as:detecting errors early during the parser phase, before spending any time running code above the line containing thesmugglestatementpreventing shell injections in onion comments—e.g.,#pip: --upgrade numpy && rm -rf /fails due to theOnionParser, but would otherwise execute successfully.allowing certain installer arguments to temporarily influencedavosbehavior while smuggling the current package
(seeInstaller options that affectdavosbehaviorbelow for specific info)Installer options that affectdavosbehaviorPassing certain options to the installer program via anonion commentwill also affect the
correspondingsmugglestatement in a predictable way:--force-reinstall|-I,--ignore-installed|-U,--upgradeThe package will be installed, even if it exists locally--no-inputDisables input prompts, analogous to temporarily settingdavos.config.noninteractivetoTrue. Overrides value
ofdavos.config.confirm_install.--src <dir>|-t,--target <dir>Prepends<dir>tosys.pathif not already present so
the package can be imported.Smuggling packages with C-extensionsSome Python packages that rely heavily on custom data types implemented viaC-extensions(e.g.,numpy,pandas) dynamically generate
modules defining various C functions and data structures, and link them to the Python interpreter when they are first
imported. Depending on how these objects are initialized, they may not be subject to normal garbage collection, and
persist despite their reference count dropping to zero. This can lead to unexpected errors when reloading the Python
module that creates them, particularly if their dynamically generated source code has been changed (e.g., because the
reloaded package is a newer version).This can occasionally affectdavos's ability tosmugglea new version of a package (or dependency) that was
previously imported. To handle this,davosfirst checks each package it installs againstsys.modules. If a different version has already been
loaded by the interpreter,davoswill attempt to replace it with the requested version. If this fails,davoswill
restore the old package versionin memory, while replacing it with the new package versionon disk. This allows
subsequent code that uses the non-reloadable module to still execute in most cases, while dependency checks for other
packages run against the updated version. Then, depending on the value ofdavos.config.auto_rerun,davoswill
either either automatically restart the interpreter to load the updated package, prompt you to do so, or raise an
exception.from...import...statements and reloading modulesThe Python docs forimportlib.reload()include
the following caveat:If a module imports objects from another module usingfrom…import…, callingreload()for the other module does
not redefine the objects imported from it — one way around this is to re-execute thefromstatement, another is to
useimportand qualified names (module.name) instead.The same applies to smuggling packages or modules from which objects have already been loaded. If objectnamefrom
modulemodulewas loaded using eitherfrom module import nameorfrom module smuggle name, subsequently
runningsmuggle module    # pip --upgradewill in fact install and load an upgraded version ofmodule, but the
thenameobject will still be that of the old version! To fix this, you can simply runfrom module smuggle nameeither instead in lieu of or aftersmuggle module.Smuggling packages from version control systemsThe first time during an interpreter session that a given package is installed from a VCS URL, it is assumed not to be
present locally, and is therefore freshly installed.pipclones non-editable VCS repositories into a temporary
directory, runssetup.py install, and then immediately deletes them. Since no information is retained about the
state of the repository at installation, it is impossible to determine whether an existing package satisfies the state
(i.e., branch, tag, commit hash, etc.) requested for smuggled package.