Package: gorpyter
GORpyterPython package (with pandas serialization) that wraps the R SDK of the GOR Query API.Docker image for JupyterLab (Python & R kernels) with both the Python & R SDK dependencies installed.gp.query()converts the R tibble dataframe into a pandas dataframe on the fly.rpy2 package is used to wrap the gorr R library functions in Python.Jupyter R kernel has tidyverse (tricky install) and gorr (non-CRAN) packages installed.Docker image also includes OpenJDK 1.8 in case users want to install Spark.TLDR$ docker pull hashrocketsyntax/gorpyter:augustus
$ docker run -it -p 8888:8888 hashrocketsyntax/gorpyter:augustusRead the rest of the documentation for complete setup & usage.1. Docker EnvironmentLOCAL NOTEBOOK FOLDERCreate a folder on your local machine's desktop where you will store your notebooks. Keep the output ofpwdhandy as we will use it with thevolumesyml key below. You can name the folder whatever you like. We'll call it 'notebooks'$ cd ~/Desktop
$ mkdir notebooks
$ cd notebooks
$ pwd
'<PATH_TO_YOUR_NEW_FOLDER>'DOCKER HARDWARE RESOURCESIn order to convert large (1M rows) R dataframes to Pandas dataframes, your Docker environment may need access to more memory. Thememoryis the most important setting below.Stop any running containers.Click on Docker icon in system tray.Navigate to 'Preferences.'Click the 'Resources' or 'Advanced' tab depending on your version of Docker.Set the resources to the following:Click 'Apply & Restart'CPU:              <keep default, should already be at 4 CPU>
Memory:           <half of what's available in 'About this Mac', 4 or 8 GB>
Swap:             <set to maximum, 4GB>
Disk Image Size:  <keep default>DOCKER IMAGE & MANIFESTPull in this pre-built image which contains a Jupyter environment equipped with R and Python 3.7 kernels as well as the GORpyter dependencies. It's built on top of Jupyter's latest DockerHub imagejupyter/datascience-notebook:2ce7c06a61a1. If you want to customize this image your self, see Section 3.$ docker pull hashrocketsyntax/gorpyter:augustusCreate a file nameddocker-compose.ymland open it with a text editor (nano or SublimeText).$ touch docker-compose.yml
$ nano docker-compose.ymlPaste the text below into that file. Under thevolumeskey, paste in the output ofpwdfrom above.#docker-compose.yml
version: "3"
services:
  jupyter:
    image: "hashrocketsyntax/gorpyter:augustus"
    ports:
      - "8888:8888"
    volumes:
      - <PATH_TO_YOUR_NEW_FOLDER>:/usr/local/share/man/user_notebooksMake sure you are in the same directory as the .yml file and run it like so.$ docker-compose upFrom the console output, grab the URL that looks like thishttp://127.0.0.1:8888/?token=<YOUR_TOKEN>and paste it into a browser.2. JupyterLab NotebooksTUTORIAL NOTEBOOKSThe Docker environment comes with example notebooks for both the Python and R SDK.If you are running these notebook in the pre-built Docker environment, know that only files in theuser_notebooksfolder will be saved/ persisted. In fact, you won't be able to add/remove/copy/delete/save-changes to files outside of theuser_notebooksdirectory.#python_sdk_gorpyter.ipynb


pip install gorpyter --upgrade
import gorpyter as gp


gp.setup()
"""
  CHECKLIST
  =============================================

	✓ -- The version of your Jupyter Python environment is '3.7.3'.
	✓ -- The path of the Jupyter R enviroment being accessed by `rpy2` is '/opt/conda/lib/R'.

	✓ -- The Python dependencies of `gorpyter` are installed.
	✓ -- The `tidyverse` R library is installed in your R environment.
	✓ -- The `gorr` R library is installed in your R environment.
	✓ -- Python was able to successfully load `gorr` as a module via `rpy2`.

  =============================================
"""


api_key = "<YOUR_API_KEY>"
project = "<YOUR_PROJECT_NAME>"
conn = gp.connect(api_key, project)


gp.query("<YOUR_GOR_QUERY>", conn)
"""
	nor example -- "nor ./"
	gor example -- "gor -p chr10 #dbsnp# | TOP 100"

	Tested successfully on a 1,000,000 row result.

	Despite being run in Python, interupting the client's execution 
  of this function in `ctrl+c` manner is surprisingly still gracefully 
  intercepted by the gorr R library, and thus the server-side 
  execution of the query is simultaneously cleaned up.
"""PYTHON PACKAGEpip install gorpyter --upgradeconda installwillnotwork as this package has not been published to conda-forge.Latest version number can be seen herehttps://pypi.org/project/gorpyter, as compared to output ofpip show gorpyter.Installing gorpyter will also install these dependencies: rpy2>=3.0.5, tzlocal>=2.0.0, pandas>=0.25.0, numpy>=1.17.0.GOR QUERY LANGUAGEhttp://docs.wuxinextcode.com/gor/basicGORqueries.html3. Optional -- Customizing the Docker ImageIn order to create your own Docker image based onjupyter/datascience-notebook:latest, follow these instructions.With these files in the same directory:Dockerfilepython_sdk.ipynbr_sdk.ipynbRundocker build -t your-image-name:your-new-tag .from within that directory.Here are the commands contained in the Dockerfile.#Dockerfile
FROM jupyter/datascience-notebook:latest
MAINTAINER layne sadler <lsadler@wuxinextcode.com>


# ====== PRE SUDO ======
ENV JUPYTER_ENABLE_LAB=yes

# If you run pip as sudo it continually prints errors.
# Tidyverse is already installed, and installing gorpyter installs the correct versions of other Python dependencies.
RUN pip install gorpyter
RUN Rscript -e "install.packages('https://cdn.nextcode.com/public/libraries/gorr_0.2.5.tar.gz', repos = NULL, type = 'source')"
ENV R_HOME=/opt/conda/lib/R

# https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch04s09.html
# Looks like /usr/local/man is symlinking all R/W toward /usr/local/share/man instead
COPY python_sdk.ipynb /usr/local/share/man
COPY r_sdk.ipynb /usr/local/share/man
ENV NOTEBOOK_DIR=/usr/local/share/man
WORKDIR /usr/local/share/man


# ====== SUDO ======
USER root

# Spark requires Java 8.
RUN sudo apt-get update && sudo apt-get install openjdk-8-jdk -y
ENV JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java

# If you COPY files into the same VOLUME that you mount in docker-compose.yml, then those files will disappear at runtime.
# `user_notebooks/` is the folder that gets mapped as a VOLUME to the user's local folder during runtime.
RUN mkdir /usr/local/share/man/user_notebooks