Package: herenow-demo
What is this?The project marries Consious Urban Exploration(it is also kind of a productization of being "Here In The Moment" and Travel) and the "new" AI for infra. Its a seed for an idea on how to explore our neighborhoods and being Here in the moment. "Here" can be at home or really anywhere you are in your city, because no matter how you got there its where you are now.Awareness and being conscious of this moment are two ideas close to my heart(Meditation is key to getting there.). I also love to travel but one can also be a "traveller" within their city(One might say we all travel on the weekend) exploring big places like NYC.I live in Hoboken, NJ so this is only content from there and intended to present a PoC.Tech stackThis repo is just the front end(FastAPI, Jinja, Bootstap CSS, Swiper.js). The API and Data pipes that uses AI(OpenAI, Pydantic, OpenRouteService) and generates the data indexed into a DB(in Sqlite!) ishereWhat it looks like?(I am about to deploy the docker to the cloud, but here are some screenshots from the local docker container to explain the idea)Landing Page: Drop a pin within the area(no location permissions requested)Next: The "Conscious Exploration" bits of :)Also importantly: The use of modern AI: A pipeline to retrieve structured info from unstructured text and use RAG(embeddings and the like) to enrich and classify the  content... So it does not appear like a smorgasbord of information.The stack that uses AI and generates the DB(in Sqlite!) ishereWhat this repo contains?The front end which calls the backend(hereand all the opsy stuff to get it onto a cloud(Ongoing).Stuff specific to building the demo(Opsy)Docs are a WIP so things may not work in the short term as expectedBuilding and Distributing the Wheel.There were a lot of package install errors(Oh debian linux how precise you are) and wrangling with the initial pyproject.toml dependencies
that needed rust/c++ etc. Often there were errors and it took a fair bit of time building them from wheels/sdists, so I removed the deps that were not needed to build these packages, cutting down immensely on the size of the final installed binaries. This took a while...Iteration Loop(code=>build=>deploy):Build the wheel of thedrop_backedproject. Update it in theherenow_demo. Both of these should be installable via pip.A thing to note about moving a dep totool.poetry.dependencies.devis thatpoetry build's wheels don't exclude any deps from the pyproject.toml in whatever section it is as explainedhere, but we can useextrasin poetry and set the optional=true on them so that later pip does not install them unless we ask.Once tested(from local docker), the builds will be tagged. We can copy them from the containers and push to pypi or we can just build and push wheels from the locally tested code.Then Use a second docker container to install these built artifacts in a docker containerPush the docker container to a registry where it can be
deployed to a k8s/ec2/gws etc cluster.Docker 1(DockerfileBuild): Tests if the wheels can be built and installed using pip:Write code, test, once done prune unwanted deps indrop_backendfrom its pyproject.toml.Run setup_drop_build to copy the code for docker build to find in its context:python -m herenow_drop.setup_drop_build --pot-source=local --demo-source=local --pot-dir=/Users/sid/workspace/drop/ --demo-dir=/Users/sid/workspace/herenow_demo/This creates a sandbox where the code for both projects is copied and then in that sandbox the script bumps the major/minor/patch version in pyproject.toml indrop_backendexamplex.y.zbecomes alphax.y.(z+1)-alpha. This will be useful later.Run docker build to build thedrop_backendproject like so:docker build --target backend-builder \
--build-arg DROP_BACKEND_DIR=drop_PoT \
--build-arg HERENOW_DEMO_DIR=herenow_demo  \
-t backend-image -f DockerfileBuild /tmp/subdir1.1. Log into docker:docker run -p 8000:8000 -it --entrypoint /bin/bash backend-image1.2 look in /backend/dist and look at the version in wheel and pyproject.toml. They should match and be incremented over the source on your local host. You can also try importing it in the python interpreter afterpip install dist/*.whlto make sure import works. We test this in the next step.Succeed? No -> Go back to step 0 and repeat.Build the app image:Copy the database to/tmp/subdir/drop.dbNow make changes to the herenow_demo app. Once done run the remaining docker build command:docker build --target app-builder \
--build-arg DROP_BACKEND_DIR=drop_PoT \
--build-arg HERENOW_DEMO_DIR=herenow_demo  \
-t app-image \
-f DockerfileBuild \
/tmp/subdirTip: When you are coding and testing locally the best way is to uncomment# drop-backend = { path = "../drop" }dep and just run uvicorn locally(poetry run herenow_demo.) to test. Just make sure the .env file has the right env vars set.4.1. Crucially, thedocker build...above runs theconfigure_demo.pyscript runs and fills in the name of the built wheel ofdrop_backendin the pyproject.toml ofherenow_demo. This is needed because the wheel name is not trivial to know until the wheel is built.4.2. It will create the wheel of herenow_demo and install it in the docker container.Did you succeed in building the image?No -> Fix the code and error on local host and go to step 4.Yes -> Go to next step.For the final step to test my web app, I used the local .env.localfile with all the environment variables set that I used for local testing for running the test server in the docker container. Modify it as needed and run:./docker_test_local_dev.shwhich will give you a final output command to build the docker image with all the environment variables set:docker build --target app-tester -t local-herenow-demo-test \
-f DockerfileBuild  \
--build-arg ALLOWED_ORIGINS=http://127.0.0.1:8000,http://localhost:8000 \
--build-arg SQLITE_DB_PATH=sqlite:////app/drop.db \
--build-arg RELOAD_WEBAPP=True \
--build-arg SECRET_KEY=secret-key \
--build-arg ORS_API_ENDPOINT=http://ors-app:8080/ors/v2/directions/{profile} \
--build-arg FILE_VERSION_CONSTRAINTS='{"hobokengirl_com_hoboken_jersey_city_events_november_10_2023_20231110_065435_postprocessed": ["v1"],"hobokengirl_com_diwali_events_hudson_county_2023_20231110_065438_postprocessed": ["v1"]}'  \
--build-arg DROP_BACKEND_DIR=drop_PoT  \
--build-arg HERENOW_DEMO_DIR=herenow_demo \
/tmp/subdirThe key is to note the args for ORS API and SQLITE DB path which are different from.env. The file_version_constraints will also change from build to build.Run the container to start the local server:docker run --network=herenow-demo -p 8000:8000 -d local-herenow-demo-testNote the network name was created so that the webapp can talk to the ORS container that might be running separately.Tail the logs to see the server running:docker logs -f -t <CONTAINER_NAME>Browse tohttp://localhost:8000/presence/whereto check.Check in the tags, preprep for docker buildNow use the tag from last step for both the projects.The tag fordrop_backendis likely:sid@Sids-MacBook-Pro:~/workspace/herenow_demo$ docker run --rm app-image  bash -c "cat /backend/pyproject.toml|grep -C2 drop_backend"

[tool.poetry]
name = "drop_backend"
version = "1.0.7-alpha" ------------------------------> This is the alpha version tag.
description = "API and Command line tools for building drop"
authors = ["Sid <itissid@gmail.com>"]
packages = [{ include = "drop_backend", from = "src" }]Add this tag to yourlocaldrop_backend's pyproject.tomlversion. Do apoetry updateandpoetry buildto make sure the wheel is built locally.Now publish publish the wheel to pypi usingpoetry publish.Use the alpha version of the wheel you just pushed for the next step. Note if you did every thing right the local dist/ should have the same file as one on docker and on pypi:sid@Sids-MacBook-Pro:~/workspace/herenow_demo$ docker run --rm app-image  bash -c "cat pyproject.toml|grep drop_backend"
file = "/backend/dist/drop_backend-1.0.7a0-py3-none-any.whl"Now use this build,1.0.7a0is the example, built above and add this to the pyproject.toml ofherenow_demoand do apoetry updateandpoetry buildto make sure the wheel is built locally. You can also bump the version ofherenow_demoitself.Buildherenow_demousingpoetry buildand push the wheel to pypi usingpoetry publishLastly add tags for these versions to git repos of both the projects and push them to github.Docker 2(DockerfileDeploy): This will be the container that will host the app and the SQLite DB.Building the app image that will be deployed on the k8s cluster is very similar to the last step for the above local buildAssuming you have updated the drop_backend version in herenow demo and published the alpha version to pypi, you can build the app image:./docker_test_before_deploy.sh --build-arg DEPLOY_TAG=1.1.0a0

$ docker build --target base -t pre-deploy-herenow-demo -f DockerfileDeploy \
--build-arg ALLOWED_ORIGINS=http://127.0.0.1:8000,http://localhost:8000 \
--build-arg SQLITE_DB_PATH=sqlite:////Users/sid/workspace/drop/drop.db \
--build-arg RELOAD_WEBAPP=True \
--build-arg SECRET_KEY=secret-key \
--build-arg ORS_API_ENDPOINT=http://orsapp:8080/ors/v2/directions/{profile} \ 
--build-arg
FILE_VERSION_CONSTRAINTS='{"hobokengirl_com_hoboken_jersey_city_events_november_10_2023_20231110_065435_postprocessed":
["v1"],"hobokengirl_com_diwali_events_hudson_county_2023_20231110_065438_postprocessed":
["v1"]}' 
--build-arg DEPLOY_TAG=1.1.0a0 .Note the DEPLOY_TAG is the version of the herenow_demo package in pypiFiring this command will give you a image with an image with the server you can test one last time before deploying:docker run -p 8000:8000 -d pre-deploy-herenow-demoCheck on the browser one last time!Now that both the deps are in the container, push to registry.Next(Deploy docker to cloud)Once this ran I deployed it to a "small" k8s cluster on Digital Ocean in two steps(my key critera was scale the app to 3 nodes in case of a spike, in case of a crash recover, and portability to any infra). The notes for that are in my openrouteservice repo that I forked.NotesWhile building the above i realized that I am actually doing a mini release strategy. Something that github or gitlab can probably help me
do, were this to ever become a product.Another cool idea is to use something like ngrok to expose the local server to the internet and then use that to test the app and show it to someone.1 Python poetry extras