Package: cellxgene-gateway
OverviewCellxgene Gateway allows you to use the Cellxgene Server provided by the Chan Zuckerberg Institute (https://github.com/chanzuckerberg/cellxgene) with multiple datasets. It displays an index of available h5ad (anndata) files. When a user clicks on a file name, it launches a Cellxgene Server instance that loads that particular data file and once it is available  proxies requests to that server.Running locallyPrequisitesThis project requires python 3.6 or higher. Please check your version with$python--versionIt is also a good idea to set up a venvpython-mvenv.cellxgene-gatewaysource.cellxgene-gateway/bin/activate# type `deactivate` to deactivate the venvInstall cellxgene-gatewayOption 1: Pip Install from Githubpipinstallgit+https://github.com/Novartis/cellxgene-gatewayNote: you may need to downgrade h5py withpip install h5py==2.9.0due to anissuein a dependency.Option 2: Install from PyPIpipinstallcellxgene-gatewayRunning cellxgene gatewayPrepare a folder with .h5ad files, for examplemkdir../cellxgene_data
wgethttps://raw.githubusercontent.com/chanzuckerberg/cellxgene/master/example-dataset/pbmc3k.h5ad-O../cellxgene_data/pbmc3k.h5adSet your environment variables correctly:exportCELLXGENE_DATA=../cellxgene_data# change this directory if you put data in a different place.exportCELLXGENE_LOCATION=`whichcellxgene`Now, execute the cellxgene gateway:cellxgene-gatewayHere's what the environment variables mean:CELLXGENE_LOCATION- the location of the cellxgene executable, e.g.~/anaconda2/envs/cellxgene/bin/cellxgeneAt least one of the following is required:CELLXGENE_DATA- a directory that can contain subdirectories with.h5addata files,withouttrailing slash, e.g./mnt/cellxgene_dataCELLXGENE_BUCKET- an s3 bucket that can contain keys with.h5addata files, e.g.my-cellxgene-data-bucketCellxgene Gateway is designed to make it easy to add additional data sources, please see the source code for gateway.py and the ItemSource interface in items/item_source.pyOptional environment variables:CELLXGENE_ARGS- catch-all variable that can be used to pass additional command line args to cellxgene serverEXTERNAL_HOST- the hostname and port from the perspective of the web browser, typicallylocalhost:5005if running locally. Defaults to "localhost:{GATEWAY_PORT}"EXTERNAL_PROTOCOL- typically http when running locally, can be https when deployed if the gateway is behind a load balancer or reverse proxy that performs https termination. Default value "http"GATEWAY_IP- ip addess of instance gateway is running on, mostly used to display SSH instructions. Defaults tosocket.gethostbyname(socket.gethostname())GATEWAY_PORT- local port that the gateway should bind to, defaults to 5005GATEWAY_EXPIRE_SECONDS- time in seconds that a cellxgene process will remain idle before being terminated. Defaults to 3600 (one hour)GATEWAY_EXTRA_SCRIPTS- JSON array of script paths, will be embedded into each page and forwarded with--scriptsto cellxgene serverGATEWAY_ENABLE_ANNOTATIONS- Set totrueor to1to enable cellxgene annotations and gene sets.GATEWAY_ENABLE_BACKED_MODE- Set totrueor to1to load AnnData in file-backed mode. This saves memory and speeds up launch time but may reduce overall performance.GATEWAY_LOG_LEVEL- default isINFO. set toDEBUGto increase logging and toWARNINGto decrease logging.S3_ENABLE_LISTINGS_CACHE- Set totrueor to1to cache listings of S3 folders for performance. If the cache becomes stale, setfilecrawl.html?refresh=truequery parameter to refresh the cache.If any of the following optional variables are set,ProxyFixwill be used.PROXY_FIX_FOR- Number of upstream proxies setting X-Forwarded-ForPROXY_FIX_PROTO- Number of upstream proxies setting X-Forwarded-ProtoPROXY_FIX_HOST- Number of upstream proxies setting X-Forwarded-HostPROXY_FIX_PORT- Number of upstream proxies setting X-Forwarded-PortPROXY_FIX_PREFIX- Number of upstream proxies setting X-Forwarded-PrefixThe defaults should be fine if you set up a venv and cellxgene_data folder as above.Running cellxgene-gateway with DockerFirst, build Docker image:dockerbuild-tcellxgene-gateway.Then, cellxgene-gateway can be launched as such:dockerrun-it--rm\-v<local_data_dir>:/cellxgene-data\-p5005:5005\cellxgene-gatewayAdditional environment variables can be provided with the-eparameter:dockerrun-it--rm\-v<local_data_dir>:/cellxgene-data\-eGATEWAY_PORT=8080\-p8080:8080\cellxgene-gatewayCustomizationThe current paradigm for customization is to modify files during a build or deployment phase:To modify CSS or JS on particular gateway pages, overwrite or append to the templatesTo add script tags such as for user analytics to all pages, set GATEWAY_EXTRA_SCRIPTSthese scripts will also be run on the pages served by cellxgene server via the --scripts parameterSeehttps://github.com/chanzuckerberg/cellxgene/pull/680for details on --scripts parameterCurrently we use a bash script that copies the gateway to a "build" directory before modifying templates with sed and the like. There is probably a better way.DevelopmentWe’re actively developing.  Please see the "future work" section of thewiki. If you’re interested in being a contributor please reach out to@alokito.Developer InstallIf you want to develop the code, you will need to clone the repo. Make sure you have the prequesite listed above, then:Clone the repogitclonehttps://github.com/Novartis/cellxgene-gateway.gitcdcellxgene-gatewayInstall requirements withpipinstall-rrequirements.txtInstall the gateway in developer modepythonsetup.pydevelopFor convenience, the code repo includes arun.sh.exampleshell script to run the gateway.Install pre-commit hookscondainstall-cconda-forgepre-commit
pre-commitinstallRunning Testspython-munittestdiscovertestsCode Coveragecoveragerun-munittestdiscovertestscoveragehtmlRunning Linterspip install isort flake8 blackisort-rc.# rc means recursive, and was deprecated in dev version of isortblack.Getting HelpIf you need help for any reason, please make a github ticket. One of the contributors should help you out.ContributorsNiket Patel -https://github.com/NiketPatel9Alok Saldanha -https://github.com/alokitoYohann Potier -https://github.com/ypotier