Package: paperspace
Paperspace PythonRelease 0.2.0Seereleasenotes.mdfor details on the current release, as well as release history.Getting StartedMake sure you have a Paperspace account set up. Go tohttp://paperspace.comto register.Use pip, pipenv, or conda to install the paperspace-python package, e.g.:pip install -U paperspaceTo install/update prerelease (Alpha/Beta) version version of paperspace-python, use:pip install -U --pre paperspaceEnable autocomplete:Add following to your.bashrc(or.zshrc) to enable autocomplete anytime you activate your shell.
If paperspace-python was installed in a virtual environment, the following has to be added to theactivatescript:eval "$(_PAPERSPACE_PYTHON_COMPLETE=source paperspace-python)"Alternatively, you can create activation script by:(_PAPERSPACE_PYTHON_COMPLETE=source paperspace-python) > ~/paperspace_complete.shand then add. ~/paperspace_complete.shto your.bashrc,.zshrcoractivatescript.More:https://click.palletsprojects.com/en/7.x/bashcomplete/Download your api key by executing the following:paperspace-python loginFollow the prompts to enter your Paperspace email and password.You can also enter your credentials directly on the command line as follows:paperspace-python login <email> <password> [<api_token_name>]Note: your api key is cached in ~/.paperspace/config.json
You can remove your cached api key by executing:paperspace-python logoutRun the sample script hello.py using Python:python hello.pyThe source of this sample script shows how a script can automatically run itself on the Paperspace job cluster node:import paperspace

paperspace.run()

print('hello paperspace-python!')Note: the source is modified before transfer to the job cluster in order to remove importedpaperspacereferences.Use paperspace-python to run a python script remotely:paperspace-python run myscript.pyThe script will be run on the Paperspace job cluster node, and its output will be logged locally.Create/create and start experimentTo create new experiment use:paperspace-python experiments create [type] [--options]The two available experiment types aresinglenodeandmultinode.To create and immediately start new experiment use:paperspace-python experiments createAndStart [type] [--options]For a full list of available commands runpaperspace experiments --help.
Note that some options are required to create new experiment.Specifying jobs options within a scriptThis example shows how a script can specify paperspace jobs options for itself, such asprojectname,machineType, and acontainerreference:# tests/test_remote.py - runs itself on paperspace, demonstrates setting jobs create options
import os
import paperspace

paperspace.run({'project': 'myproject', 'machineType': 'P5000',
                'container': 'paperspace/tensorflow-python'})

print(os.getcwd())
print('something useful')Automatic running of a python script remotelyThe above example demonstrates running a python script locally and having that script transmit itself to the paperspace jobs cluster for further execution.  To do this a copy of the local script is modified before transmission to the jobs cluster, in order to strip out theimport paperspacestatements and otherpaperspacelibrary references. There are also some limitations on the types of import statements that are supported, and the dependencies that are supported in each environment (local vs. remote):You need to use a bare import statement,import paperspace, and not use theimport paperspace as ...form.The import formfrom paperspace import ...is currently not supported.Everything after thepaperspace.run()function call is ignored when running locally (when no script name is provided).  The local script execution stops after thepaperspace.run()call.Dependencies that are included beforepaperspace.run()must be available locally.If you need to reference dependencies that are not available locally but are available remotely, those should be imported after thepaperspace.run()call.Dependencies that are needed remotely need to either be already installed in the container used for the job, or need to be installed using one of the techniques below in the sectionDependency OptionsBecause of these limitations it may not always be appropriate to run python scripts automatically from within the same script file.  As an alternative you can run your python scripts unmodified using the techniques below.Running a python script by nameYou can run an python script on paperspace from the command line as follows:paperspace-python run myscript.pyYou can also provide additional jobs options on the command line:paperspace-python run myscript.py --project myproject --machineType P5000 \
 --container paperspace/tensorflow-python`Alternatively you can use thepaperspace.run()fuction in code with a script file name as the first argument:import paperspace

paperspace.run('myscript.py') # runs myscript on paperspaceIn code you can provide additional paperspace jobs create options in a dict in the second argument to run():paperspace.run('myscript.py', {'project': 'myproject', 'machineType': 'P5000',
                               'container': 'paperspace/tensorflow-python'})See the Paperspace APIjobs createdocumentation for the full list of jobs create options that can be specified.Using paperspace-python runThepaperspace-python runcommand provides a number of options to run python code and other commands remotely, as well as copy files and set up python dependencies:paperspace-python run [options] [[-m] <script> [args] | -c "python code" | --command "shell cmd"]
  options:
  [--python 2|3]
  [--init [<init.sh>]]
  [--pipenv]
  [--req [<requirements.txt>]]
  [--workspace .|<workspace_path>]
  [--ignoreFiles "<file-or-dir>,..."]
  [jobs create options]
  [--dryrun]
  [-]Basic Run ScenariosRun a python script remotely:paperspace-python run <python_script.py> [args]Example:paperspace-python run myscript.py a b cRun a python module remotely using the-moption:paperspace-python run -m <module_path> [args]Example:paperspace-python run -m pip --versionRun a python command remotely using the-coption:paperspace-python run -c "python_statement;..."Example:paperspace-python run -c "import os; print(os.getcwd())"Run an executable or shell command remotely using the--commandoption:paperspace-python run --command "<executable or shell command>"Example:paperspace-python run --command "ls -al"Run OptionsThe<script>option is a python script or path to a python module.  The script or module will be uploaded if it exists on the local file system.Other scriptargscan be provided after the python script or module path.  You can use the-option to suppress interpretation of the list of script args aspaperspace-python runoptions.The-m <module path>option runs the specified library module as a script.  This is equivalent to the-moption of thepythonexecutable.  Further paperspace run option processing is disabled after the-moption.The-c "python_statement;..."option runs the specified python statements.  This is equivalent to the-coption of thepythonexecutable.  Further paperspace run option processing is disabled after the-coption.The-option disables further run command option processing and passes the remaining arguments to the script specified.  This allows you to pass arguments to your script that might otherwise conflict with run command options or jobs create options.The--command "shell cmd"option is used to run an arbitrary executable or shell command inside the container.  Note: the executable or shell command must already be available inside the container image, or be copied over using the--workspaceoption.Job OptionsThe--workspaceoption allows you to specify a workspace file or directory to upload, or a git repo link to download and merge with the container.  For example, to upload the current directory along with a script file run:paperspace-python run myscript.py --workspace .See the Paperspae APIjobs createdocumentation for more details on the--workspaceoption and related options.The--ignoreFiles "<file-or-dir>,..."option can be used specify a simple comma separated list of files and directories to ignore for the workspace upload:paperspace-python run myscript.py --workspace . --ignoreFiles "hello.py,paperspace"The following files and directories are ignored by default:.git,.gitignore,__pycache__.Otherjobs create optionscan be specified, such as--machineType <machine type>,--container <container image reference>, and--project <project name>.Here are some of the other jobs create options available:--project "<project name>"(defaults to 'paperspace-python')--machineType [GPU+|P4000|P5000|P6000|V100](defaults to P5000)--container <docker image link or paperspace container name>(defaults todocker.io/paperspace/tensorflow-python)--name "<job name>"(defaults to 'job for project ')--projectId "<existing paperspace project id>"--registryUsername "<username>"(for access to a private docker registry)--registryPassword "<secretpw>"(for access to a private docker registry)--workspaceUsername "<username>"(for access to a private git repo)--workspacePassword "<secretpw>"(for access to a private git repo)See the Paperspae APIjobs createdocumentation for a complete description of these options.Dependency OptionsWhen running python scripts on paperspace you may need to provide additional dependencies to your scripts or specify the python version.
Thepaperspace-python runcommand has several options to support this:--python,--init,--pipenv, and--req. In addition you can use the--workspaceoption above to upload file dependencies.The--python 2|3option allows you specify whether to usepython2orpython3when running the script on paperspace.
If ommitted, the script will be run with the same major version as is being used to runpaperspace-pythonlocally.The--init [<init.sh>]option is used to specify a script to be run on the remote machine, inside the container, before the python script is run.
If the init script name is ommitted, it is assumed to be the script namedinit.shin the current directory.  The script is run usingsource init.shin the container bash shell.  You can use this option to provide a list of commands to run to set up the dependencies for the script, such as running a list ofpip installcommands.  However, if you are usingpipenvor arequirements.txtfile we recommend you use one of the options below.  Multiple dependency setup options can be combinded however.The--pipenvoption is used to upload and run thePipfileandPipfile.lockfiles in the current directory, if found.  These files
are used on the paperspace machine to initialize the python environment using thepipenvtool, by runningpipenv installwithin the container.  Note:pipenvmust already be installed in the container for this option to work.  The default container used by paperspace-python already has thepipenvpackage installed.The--req [<requirements.txt>]option is used specify that arequirements.txtfile should be used to install the required python dependencies usingpip.
By default this option looks for a file namedrequirements.txtin the current directory, but you can override this by specifying a different file name.  Note:pipmust already be installed in the container for this option to work.  The default container used by paperspace-python already has thepippackage installed.The--dryrunoption allows you to see the resultant script that will be run on the paperspace job runner without actually running it.All of the above options can be combined in any combination, however, the order of operations is fixed to the following:source <init.sh>is run if--init <init.sh>is specifiedpipenv [--two|--three] installis run if--pipenvis specifiedpip[2|3] install -r requirements.txtis run if--req <requirements.txt>is specifiedpython[2|3] myscript.pyis runAs mentioned above, you can use the--dryrunoption to see the resultant commands that will be run on the paperspace jobs cluster node for a given set of options, without actually running the commands.Default ContainerIf nocontaineroption is specified when usingpaperspace run <script.py>or thepaperspace.run()function the default container image used ispaperspace/tensorflow-pythonon Docker Hub.  This container has the tensorflow-gpu libraries installed for both python2 and python3, as well as several other popular packages, including numpy, scipy, scikit-learn, pandas, Pillow and matplotlib.
It is based off the Google docker imagegcr.io/tensorflow/tensorflow:1.5.0-gpuwith the addition of support for python3, pip3, and pipenv.A Dockerfile for building this container image ishere.Other examplesSee the scripts in thetestsfolder for other examples.Other Authentication optionsSpecify your apiKey explicitly on the paperspace.run() function or any of the paperspace.jobs methods, e.g.:paperspace.jobs.create({'apiKey': '1qks1hKsU7e1k...', 'project': 'myproject',
                        'machineType': 'P5000', 'container': 'paperspace/tensorflow-python'})Set the package paperspace.config option in your python code:paperspace.config.PAPERSPACE_API_KEY = '1qks1hKsU7e1k...'Set the PAPERSPACE_API_KEY environment variable:(on linux/mac:)export PAPERSPACE_API_KEY=1qks1hKsU7e1k...(on windows:)set PAPERSPACE_API_KEY=1qks1hKsU7e1k...Note: the above methods take precedence over use of the cached api key in~/.paperspace/config.jsonSet the key in the~/.paperspace/config.jsonfile from the command line by running:paperspace-python apikey 1qks1hKsU7e1k...Using SAML, AD or GitHub credentialsCurrently only email login is supported in the CLI - if you're using AD, SAML or GitHub to login to Paperspace, you will need an API key to log in with the CLI.You can create an API key from within your Paperspace console under theAPIsection. Login to yourPaperspace console, scroll to the API section in the left navigation bar, and clickCREATE AN API KEY. Follow the instructions there.
You will need to pick and API token name for your API key, and also provide a description.  You can copy the API key value associated with the API token name only at the time of initial creation. If you need to access your API key in the future, you can instead access it by API token name using the 'paperspace-python login' command.ContributingWant to contribute?  Contact us athello@paperspace.com