Package: dlunch
Data LunchThe ultimate web app for a well organized lunch.1. Table of contents1. Table of contents2. Development environment setup2.1. Miniconda2.2. Setup the development environment2.3. Environment variables2.3.1. General2.3.2. Docker and Google Cloud Platform2.3.3. TLS/SSL Certificate2.3.4. Encryption and Authorization2.4. Manually install the development environment2.5. Manually install data-lunch CLI2.6. Running the docker-compose system2.7. Running a single container2.8. Running locally3. Additional installations before contributing3.1. Pre-commit hooks3.2. Commitizen4. Release strategy fromdevelopmenttomainbranch5. Google Cloud Platform utilities2. Development environment setupThe following steps will guide you through the installation procedure.2.1. MinicondaCondais required for creating the development environment (it is suggested to installMiniconda).2.2. Setup the development environmentUse the setup script (setup_dev_env.sh) to install all the required development tools.Usesourceto properly launch the script.source setup_dev_env.sh[!IMPORTANT]
The setup script will take care of setting up the development environment for you.
The script installs:3 environments (data-lunchfor development,ci-cdforpre-commitand other utilities,gc-sdkfor interacting withGoogle Cloud Platform)pre-commithooksdata-lunchcommand line2.3. Environment variablesThe following environment variables are required for running theweb app, themakefileorutility scripts.2.3.1. GeneralVariableTypeRequiredDescriptionPANEL_APPstr✔️app name,data-lunch-appby default (used bymakefile)PANEL_ENVstr✔️environment, e.g.development,quality,production, affects app configuration (Hydra) and build processes (makefile)PANEL_ARGSstr❌additional arguments passed toHydra(e.g.panel/gui=major_release) inmakefileanddocker-composecommandsPORTint✔️port used by the web app (or the container), default to5000; affects app configuration and build process (it is used bymakefile,HydraandDocker)2.3.2. Docker and Google Cloud Platform[!NOTE]
The following variables are mainly used during the building process or by external scripts.VariableTypeRequiredDescriptionDOCKER_USERNAMEstr❌yourDocker Hubusername, used bymakefileand stats panel to extract container nameIMAGE_VERSIONstr❌Dockerimage version, typicallystableorlatest(used bymakefileanddocker-composecommands)GCLOUD_PROJECTstr❌Google Cloud Platformproject_id, used bymakefileforGCP's CLIauthentication and for uploading the database togcpstorage, if active in web app configuration files (see panel.scheduled_tasks)GCLOUD_BUCKETstr❌Google Cloud Platformbucket, used for uploading database togcpstorage, if active in web app configuration files (see panel.scheduled_tasks)MAIL_USERstr❌email client user, used for sending emails containing the instance IP, e.g.mywebappemail@email.com(used only forGoogle Cloud Platform)MAIL_APP_PASSWORDstr❌email client password used for sending emails containing the instance IP (used only forGoogle Cloud Platform)MAIL_RECIPIENTSstr❌email recipients as string, separated by,(used for sending emails containing the instance IP when hosted onGoogle Cloud Platform)DUCKDNS_URLstr❌URLused incompose_init.shto update dynamic address (seeDuck DNS'sinstructions for details, used when hosted onGoogle Cloud Platform)2.3.3. TLS/SSL Certificate[!TIP]
Use the commandmake ssl-gen-certificateto generate local SSL certificates.VariableTypeRequiredDescriptionCERT_EMAILstr❌email for registeringSSL certificates, shared with the authorityLet's Encrypt(viacertbot); used bydocker-composecommandsDOMAINstr❌domain name, e.g.mywebapp.com, used bydocker-composecommands (certbot), in email generation (scriptsfolder) and to auto-generate SSL certificates2.3.4. Encryption and Authorization[!NOTE]
All variables used byPostgresqlare not required ifdb=sqlite(default value).All variables used byOAuthare not required ifserver=no_auth(default value).[!IMPORTANT]DATA_LUNCH_COOKIE_SECRETandDATA_LUNCH_OAUTH_ENC_KEYare required even ifserver=no_authis set.VariableTypeRequiredDescriptionDATA_LUNCH_COOKIE_SECRETstr✔️Secretused for securing the authentication cookie (usemake generate-secretsto generate a valid secret)DATA_LUNCH_OAUTH_ENC_KEYstr✔️Encription keyused by the OAuth algorithm for encryption (usemake generate-secretsto generate a valid secret)DATA_LUNCH_OAUTH_KEYstr❌OAuth keyused for configuring the OAuth provider (GitHub,Azure)DATA_LUNCH_OAUTH_SECRETstr❌OAuth secretused for configuring the OAuth provider (GitHub,Azure)DATA_LUNCH_OAUTH_REDIRECT_URIstr❌OAuth redirect uriused for configuring the OAuth provider (GitHub,Azure), do not set to use default valueDATA_LUNCH_OAUTH_TENANT_IDstr❌OAuth tenant idused for configuring the OAuth provider (Azure), do not set to use default valueDATA_LUNCH_DB_USERstr❌Postgresqluser, do not set to use default valueDATA_LUNCH_DB_PASSWORDstr❌PostgresqlpasswordDATA_LUNCH_DB_HOSTstr❌Postgresqlhost, do not set to use default valueDATA_LUNCH_DB_PORTstr❌Postgresqlport, do not set to use default valueDATA_LUNCH_DB_DATABASEstr❌Postgresqldatabase, do not set to use default valueDATA_LUNCH_DB_SCHEMAstr❌Postgresqlschema, do not set to use default value2.4. Manually install the development environment[!WARNING]
This step is not required if thesetup script(setup_dev_env.sh) is used.Use the terminal for navigating to the repository base directory.Use the following command in your terminal to create an environment nameddata-lunchmanually.Otherwise use thesetup scriptto activate the guided installing procedure.conda env create -f environment.ymlActivate the newCondaenvironment with the following command.conda activate data-lunch2.5. Manually install data-lunch CLI[!WARNING]
This step is not required if thesetup script(setup_dev_env.sh) is used.The CLI is distributed with setuptools instead of using Unix shebangs.It is a very simple utility to initialize and delete the app database. There are different use cases:Create/delete thesqlitedatabase used by the appInitialize/drop tables inside thesqlitedatabaseUse the following command for generating the CLI executable from thesetup.pyfile, it will install your package locally.pip install .If you want to make some changes to the source code it is suggested to use the following option.pip install --editable .It will just link the package to the original location, basically meaning any changes to the original package would reflect directly in your environment.Now you can activate theCondaenvironment and access theCLIcommands directly from the terminal (without using annoyingshebangsor prependingpythonto run yourCLIcalls).Test that everything is working correctly with the following commands.data-lunch --version
data-lunch --help2.6. Running the docker-compose systemSince this app will be deployed with an hosting service aDockerfileto build a container image is available.The docker compose file (seedocker-compose.yaml) deploys the web app container along with aload balancer(thenginxcontainer)
to improve the system scalability.Look inside themakefileto see thedockeranddocker-composeoptions.To build and run the dockerized system you have to installDocker.Call the followingmakecommand to start the build process.make docker-up-init docker-up-buildup-initinitialize thessl certificatebased on the selected environment (as set in the environment variablePANEL_ENV, i.e.developmentorproduction).Call onlymake(without arguments) to trigger the same command.A missing or incompletessl certificate folderwill result in annginxcontainer failure on start-up.Images are built and the two containers are started.You can then access your web app athttp://localhost:PORT(wherePORTwill match the value set through the environment variable).[!NOTE]
You can also usemake docker-upto spin up the containers if you do not need to re-build any image or initialize ssl certificate folders.[!IMPORTANT]
An additional container nameddbis started ifdb=postgresqlis set2.7. Running a single containerIt is possible to launch a single server by calling the following command.make docker-build

make docker-runYou can then access your web app athttp://localhost:5000(if the deafultPORTis selected).2.8. Running locallyLaunch a local server with default options by calling the following command.python -m dlunchUseHydraarguments to alter the app behaviour.python -m dlunch server=basic_authSeeHydra's documentationfor additional details.3. Additional installations before contributing[!WARNING]
This step is not required if thesetup script(setup_dev_env.sh) is used.Before contributing please create thepre-commitandcommitizenenvironments.cd requirements
conda env create -f pre-commit.yml
conda env create -f commitizen.yml3.1. Pre-commit hooksThis step is not required if thesetup scriptis used.Then install the precommit hooks.conda activate pre-commit
pre-commit install
pre-commit autoupdateOptionally run hooks on all files.pre-commit run --all-files3.2. Commitizen[!WARNING]
This step is not required if thesetup script(setup_dev_env.sh) is used.TheCommitizenhook checks that rules forconventional commitsare respected in commits messages.
Use the following command to enjoyCommitizen'sinteractive prompt.conda activate commitizen
cz commitcz cis a shorther alias forcz commit.4. Release strategy fromdevelopmenttomainbranch[!CAUTION]
This step is required only if the CI-CD pipeline onGitHubdoes not work.In order to take advantage ofCommitizenbumpcommand follow this guideline.First check that you are on the correct branch.git checkout mainThen start the merge process forcing it to stop before commit (--no-commit) and without using thefast forwardstrategy (--no-ff).git merge development --no-commit --no-ffCheck that results match your expectations then commit (you can leave the default message).git commitNowCommitizenbumpcommand will add an additional commit with updated versions to every file listed insidepyproject.toml.cz bump --no-verifyYou can now merge results of the release process back to thedevelopmentbranch.git checkout development
git merge main --no-ffUse"update files from last release"or the default text as commit message.5. Google Cloud Platform utilities[!WARNING]
This step is not required if thesetup script(setup_dev_env.sh) is used.The makefile has two rules for conviniently setting up and removing authentication credentials forGoogle Cloud Platformcommand line interface:gcp-configandgcp-revoke.