Repository: AlertaDengue/AlertaDengue (language: python)

Issue:
Upgrade postgresql 9.6 to 14
Update database version in staging and production containers.


Discussion / hints:

Fix patch:
diff --git a/.dockerignore b/.dockerignore
index 35f6c302..4bd3955d 100644
--- a/.dockerignore
+++ b/.dockerignore
@@ -1,6 +1,6 @@
 ci/*
 docs/*
-docker/postgres/dumps/*
+containers/postgres/dumps/*
 notebooks_tmp/*
 notebooks/*
 rever/*
diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml
index 217ee255..ba6e4b19 100644
--- a/.github/workflows/linux.yml
+++ b/.github/workflows/linux.yml
@@ -79,22 +79,22 @@ jobs:
 
     - name: Download data files from Dataverse
       run: |
-        mkdir -p docker/postgres/dumps
+        mkdir -p containers/postgres/dumps
         tree -L 1 ../
-        wget https://dataverse.harvard.edu/api/access/datafile/4154429 -O docker/postgres/dumps/latest_infodengue.sql.gz
-        wget https://dataverse.harvard.edu/api/access/datafile/5110797 -O docker/postgres/dumps/latest_dengue.sql.gz
+        wget https://dataverse.harvard.edu/api/access/datafile/4154429 -O containers/postgres/dev_dumps/latest_infodengue.sql.gz
+        wget https://dataverse.harvard.edu/api/access/datafile/5110797 -O containers/postgres/dev_dumps/latest_dengue.sql.gz
 
     - name: Build docker images
       run: |
-         make docker-build
+         make container-build
 
     - name: Docker start web
       run: |
-        make docker-run-all
+        make container-run-all
 
     - name: Docker waiting services
       run: |
-        make docker-wait-all
+        make container-wait-all
 
     - name: Linting
       run: |
diff --git a/.gitignore b/.gitignore
index f9347000..f5bd876b 100644
--- a/.gitignore
+++ b/.gitignore
@@ -35,7 +35,6 @@ AlertaDengue/static/*
 /AlertaDengue/*.dbf
 /*.dbf
 *.egg-info
-/docker/dev_dumps/
 /dist/
 /build/
 /tiffs/
@@ -52,4 +51,5 @@ maintenance_mode_state.txt
 rever/
 
 #dump
-docker/postgres/dumps/*
+*latest_dengue.sql.gz
+containers/postgres/setup/dumps/*
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 77ee37d9..bbcdb0fd 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -10,21 +10,21 @@
 
 ### Setting up the environment
 
- 1) Fork and clone both repositories:           
-        > AlertaDengue: https://github.com/AlertaDengue/AlertaDengue            
+ 1) Fork and clone both repositories:
+        > AlertaDengue: https://github.com/AlertaDengue/AlertaDengue
         > Data: https://github.com/AlertaDengue/Data (Contains randomly generated data for testing and development.)
-        
+
  2) Configure Demo Database in your local machine:
- 
+
      Follow [Data/README.md](https://github.com/AlertaDengue/Data#readme) to build and deploy the Demo Database image.
 
  3) Open the repository AlertaDengue with your IDE and use Miniforge ([see documentation](https://github.com/conda-forge/miniforge)) to create and activate the working environment ```(alertadengue-dev)```:
- 
+
         $ mamba env create -f conda/environment.yaml
-        $ conda activate alertadengue-dev 
+        $ conda activate alertadengue-dev
 
  4) Install the environment dependencies with Poetry ([see documentation](https://python-poetry.org/docs/)):
-         
+
          $ poetry install
 
 ### Building the app
@@ -36,20 +36,20 @@ With the environment dependencies set, it's time to prepare it to deploy. Use th
         $ make prepare-env
 
  2) Synchronize all Map and Static Files:
-         
+
         $ make sync-static-geofiles
 
  3) Build the app containers:
-         
-        $ make docker-build
-    
+
+        $ make container-build
+
     _This step will possibly fail if the .env file isn't properly filled. Please check if there isn't any empty attribute that is stopping the build._
 
  4) After the build, start the Docker images with the command:
-        
-        $ make docker-start
 
-        
+        $ make container-start
+
+
 ### Setting up Git, GitHub
 
 In order to push your work to the main project, you will need to configure the remotes Upstream and Origin within your local repository following the steps below, this will ensure you are working with the most updated version of the project. _For more details about commiting, check the Git Guides [here](https://github.com/git-guides)._
@@ -59,15 +59,15 @@ In order to push your work to the main project, you will need to configure the r
 
         $ git remote add origin git@github.com:<user>/AlertaDengue.git
         $ git remote add upstream git@github.com:AlertaDengue/AlertaDengue.git
-        
+
 2) Fetch all branches:
 
         $ git fetch --all
-        
+
 3) Commit & Create pull request
 
-    When the changes in your branch are done (_please, read about [Best Practices](https://gist.github.com/luismts/495d982e8c5b1a0ced4a57cf3d93cf60) with Git before commiting_), it's time to push the commits to your fork and create a pull request so the maintainers can review and merge into the AlertaDengue repository. 
-        
+    When the changes in your branch are done (_please, read about [Best Practices](https://gist.github.com/luismts/495d982e8c5b1a0ced4a57cf3d93cf60) with Git before commiting_), it's time to push the commits to your fork and create a pull request so the maintainers can review and merge into the AlertaDengue repository.
+
         $ git push
-        
-    Your IDE should redirect you to the GitHub page with your Pull Request specifying the commits and the files that you have worked on. Write a message telling us about all your changes and click on Create Pull Request. 
+
+    Your IDE should redirect you to the GitHub page with your Pull Request specifying the commits and the files that you have worked on. Write a message telling us about all your changes and click on Create Pull Request.
diff --git a/Makefile b/Makefile
index 94e39ff4..28628102 100644
--- a/Makefile
+++ b/Makefile
@@ -1,5 +1,7 @@
-.ONESHELL:
-
+#* Variables
+SHELL:=/usr/bin/env bash
+ARGS:=
+CONSOLE:=bash
 include .env
 
 SERVICES:=
@@ -7,11 +9,11 @@ SERVICE:=
 # options: dev, prod
 ENV:=$(ENV)
 
-DOCKER=docker-compose \
+CONTAINER_APP=docker-compose \
 	--env-file .env \
 	--project-name infodengue-$(ENV) \
-	--file docker/compose-base.yaml \
-	--file docker/compose-$(ENV).yaml
+	--file containers/compose-base.yaml \
+	--file containers/compose-$(ENV).yaml
 
 CONSOLE:=bash
 
@@ -29,88 +31,89 @@ sync-static-geofiles:
 	python AlertaDengue/manage.py sync_geofiles
 	python AlertaDengue/manage.py collectstatic --noinput
 
-# DOCKER
-.PHONY:docker-build
-docker-build:
-	$(DOCKER) build base
-	$(DOCKER) build ${SERVICES}
+# CONTAINER_APP
+
+.PHONY:container-build
+container-build:
+	$(CONTAINER_APP) build base
+	$(CONTAINER_APP) build ${SERVICES}
 
-.PHONY:docker-start
-docker-start:
-	$(DOCKER) up -d ${SERVICES}
+.PHONY:container-start
+container-start:
+	$(CONTAINER_APP) up -d ${SERVICES}
 
-.PHONY:docker-stop
-docker-stop:
-	$(DOCKER) stop ${SERVICES}
+.PHONY:container-stop
+container-stop:
+	$(CONTAINER_APP) stop ${SERVICES}
 
-.PHONY:docker-exec
-docker-exec:
-	$(DOCKER) exec ${SERVICES} bash
+.PHONY:container-exec
+container-exec:
+	$(CONTAINER_APP) exec ${SERVICES} bash
 
-.PHONY:docker-restart
-docker-restart: docker-stop docker-start
+.PHONY:container-restart
+container-restart: container-stop container-start
 	echo "[II] Docker services restarted!"
 
-#* Test docker build Ci
-.PHONY: docker-run-all
-docker-run-all: ## build and deploy all containers
-	$(DOCKER) up -d --build
+#* Test containers build on Ci
+.PHONY: container-run-all
+container-run-all: ## build and deploy all containers
+	$(CONTAINER_APP) up -d --build
 
-.PHONY:docker-logs
-docker-logs:
-	$(DOCKER) logs --tail 300 ${SERVICES}
+.PHONY:container-logs
+container-logs:
+	$(CONTAINER_APP) logs --tail 300 ${SERVICES}
 
-.PHONY:docker-logs-follow
-docker-logs-follow:
-	$(DOCKER) logs --follow --tail 300 ${SERVICES}
+.PHONY:container-logs-follow
+container-logs-follow:
+	$(CONTAINER_APP) logs --follow --tail 300 ${SERVICES}
 
-.PHONY: docker-wait
-docker-wait:
-	ENV=${ENV} timeout 90 ./docker/scripts/healthcheck.sh ${SERVICE}
+.PHONY: container-wait
+container-wait:
+	ENV=${ENV} timeout 90 ./containers/scripts/healthcheck.sh ${SERVICE}
 
-.PHONY: docker-wait-all
-docker-wait-all:
-	$(MAKE) docker-wait ENV=${ENV} SERVICE="memcached"
-	$(MAKE) docker-wait ENV=${ENV} SERVICE="rabbitmq"
-	if [[ ${ENV} -eq "dev"]]; then $(MAKE) docker-wait ENV=${ENV} SERVICE="db"; fi
-	$(MAKE) docker-wait ENV=${ENV} SERVICE="web"
-	$(MAKE) docker-wait ENV=${ENV} SERVICE="worker"
+.PHONY: container-wait-all
+container-wait-all:
+	$(MAKE) container-wait ENV=${ENV} SERVICE="memcached"
+	$(MAKE) container-wait ENV=${ENV} SERVICE="rabbitmq"
+	$(MAKE) container-wait ENV=${ENV} SERVICE="web"
+	$(MAKE) container-wait ENV=${ENV} SERVICE="worker"
+	$(MAKE) container-wait ENV=${ENV} SERVICE="db"
 
-.PHONY:docker-console
-docker-console:
-	$(DOCKER) exec ${SERVICE} ${CONSOLE}
+.PHONY:container-console
+container-console:
+	$(CONTAINER_APP) exec ${SERVICE} ${CONSOLE}
 
-.PHONY:docker-down
-docker-down:
-	$(DOCKER) down --volumes
+.PHONY:container-down
+container-down:
+	$(CONTAINER_APP) down --volumes
 
 .PHONY:run-dev-db
-docker-run-dev-db:
-	$(DOCKER) run --rm db postgres -V
+container-run-dev-db:
+	$(CONTAINER_APP) run --rm db postgres -V
 
 # Migrate databases and create shapefiles to synchronize with static_files
 .PHONY:django-migrate
-django-migrate: docker-run-dev-db
-	$(DOCKER) run --rm web python3 manage.py migrate --database=dados --noinput
-	$(DOCKER) run --rm web python3 manage.py migrate --database=infodengue --noinput
-	$(DOCKER) run --rm web python3 manage.py migrate forecast --database=forecast
+django-migrate: container-run-dev-db
+	$(CONTAINER_APP) run --rm web python3 manage.py migrate --database=dados --noinput
+	$(CONTAINER_APP) run --rm web python3 manage.py migrate --database=infodengue --noinput
+	$(CONTAINER_APP) run --rm web python3 manage.py migrate forecast --database=forecast
 
 .PHONY:django-static-geofiles
 django-static-geofiles:
-	$(DOCKER) run --rm web python3 manage.py sync_geofiles
-	$(DOCKER) run --rm web python3 manage.py collectstatic --noinput
+	$(CONTAINER_APP) run --rm web python3 manage.py sync_geofiles
+	$(CONTAINER_APP) run --rm web python3 manage.py collectstatic --noinput
 
 .PHONY:test-staging-web
 test-staging-web:
-	$(DOCKER) run --no-deps web bash /opt/services/test.sh dados
-	$(DOCKER) run --no-deps web bash /opt/services/test.sh dbf
-	$(DOCKER) run --no-deps web bash /opt/services/test.sh gis
-	$(DOCKER) run --no-deps web bash /opt/services/test.sh api
-	#$(DOCKER) run --no-deps web bash /opt/services/test.sh forecast
+	$(CONTAINER_APP) run --no-deps web bash /opt/services/test.sh dados
+	$(CONTAINER_APP) run --no-deps web bash /opt/services/test.sh dbf
+	$(CONTAINER_APP) run --no-deps web bash /opt/services/test.sh gis
+	$(CONTAINER_APP) run --no-deps web bash /opt/services/test.sh api
+	#$(CONTAINER_APP) run --no-deps web bash /opt/services/test.sh forecast
 
 .PHONY:test-staging-all
 test-staging-all:
-	$(DOCKER) run --rm web python3 manage.py test
+	$(CONTAINER_APP) run --rm web python3 manage.py test
 
 
 .PHONY: lint
@@ -121,7 +124,7 @@ lint: ## formatting linter with poetry
 # [CRON] Uses for web services
 .PHONY:send-mail-partner
 send-mail-partner:
-	$(DOCKER) run --rm web python manage.py send_mail
+	$(CONTAINER_APP) run --rm web python manage.py send_mail
 
 # Python
 .PHONY: clean
diff --git a/README.md b/README.md
index aa143d0a..5f4c597c 100644
--- a/README.md
+++ b/README.md
@@ -1,6 +1,6 @@
-# AlertaDengue
+## AlertaDengue
 
-## This repository contains the main applications and services for the InfoDengue web portal.
+### This repository contains the main applications and services for the InfoDengue web portal.
 
 ### InfoDengue is an early-warning system to all states of Brazil, the system is based on the continuous analysis of hybrid data generated through the research of climate and epidemiological data and social scraping.
 
@@ -9,11 +9,12 @@ For more information, please visit our website [info.dengue.mat.br](https://info
 
 ---
 
-### Check out below the softwares we use in the project:
+### Sponsors
 
-| <a href=https://www.djangoproject.com/><img width="298" height="100%" alt="Django" src="https://i.imgur.com/Z9wo3bS.png"></a> | <a href=https://postgis.net/documentation/><img width="298" height="100%" alt="postgis" src="https://i.imgur.com/pVEX2Gl.png">|<a href=https://docs.docker.com/get-started/><img width="298" height="100%" alt="docker" src="https://i.ibb.co/Yp8B38R/docker.png"> |
-|:-------------------------:|:-------------------------:|:-------------------------:|
-|<a href=https://docs.celeryq.dev/en/stable/><img width="298" height="100%" alt="celery" src="https://i.ibb.co/L81p2zD/celery.png"> | <a href=https://docs.nginx.com/><img width="298" height="100%" alt="nginx" src="https://i.ibb.co/2n5HZBg/nginx.png">|<a href=https://plotly.com/><img width="298" height="100%" alt="plotly" src="https://i.ibb.co/r0HYsYH/plotly.png">|
+<div style="width: 100%; text-align: left; position: relative;">
+    <a href="https://portal.fiocruz.br/"> <img width="250" alt="Fiocruz" src="https://institutolula.org/uploads/6862.png" />
+    <a href="https://emap.fgv.br/"> <img width="350" src="https://i.imgur.com/bdfNqNv.png" alt="FGV EMAp" align=right />
+</div>
 
 ---
 
@@ -22,9 +23,9 @@ For more information, please visit our website [info.dengue.mat.br](https://info
 You can find more information about [Contributing](https://github.com/AlertaDengue/AlertaDengue/blob/main/CONTRIBUTING.md) on GitHub. Also check our [Team](https://info.dengue.mat.br/equipe/) page to see if there is a work oportunity in the project.
 
 ---
-## How data can be visualized 
+## How data can be visualized
 
-The Infodengue website is accessed by many people and it is common for us to receive news that this information is used in the definition of travel and other activities. All data is compiled, analyzed and generated in a national level with the support of the Brazilian Health Ministry, the weekly reports can be found in our website through graphics or downloaded in JSON and CSV files via [API](https://info.dengue.mat.br/services/api).  
+The Infodengue website is accessed by many people and it is common for us to receive news that this information is used in the definition of travel and other activities. All data is compiled, analyzed and generated in a national level with the support of the Brazilian Health Ministry, the weekly reports can be found in our website through graphics or downloaded in JSON and CSV files via [API](https://info.dengue.mat.br/services/api).
 
 
 ### API
@@ -37,7 +38,7 @@ If you are a member of a Municipal Health Department, or a citizen, and you have
 
 ---
 
-## Where the data comes from 
+## Where the data comes from
 - Dengue, Chikungunya and Zika data are provided by [SINAN](http://portalsinan.saude.gov.br/) as a notification form that feeds a municipal database, which is then consolidated at the state level and finally, federally by the Ministry of Health. Only a fraction of these cases are laboratory confirmed, most receive final classification based on clinical and epidemiological criteria. From the notified cases, the incidence indicators that feed the InfoDengue are calculated.
 - InfoDengue has partnered with the [Dengue Observatory](https://www.observatorio.inweb.org.br/), that captures and analyzes tweets from geolocalized people for the mention of dengue symptoms on social media.
 - Weather and climate data are obtained from [REDEMET](https://www.redemet.aer.mil.br/) in the airports all over Brazil.
@@ -45,13 +46,10 @@ If you are a member of a Municipal Health Department, or a citizen, and you have
 
 ---
 
-### Sponsors
+### Check out below the softwares we use in the project:
 
-<div style="width: 100%; text-align: left; position: relative;">
-    <a href="https://portal.fiocruz.br/"> <img width="250" alt="Fiocruz" src="https://institutolula.org/uploads/6862.png" />
-    <a href="https://emap.fgv.br/"> <img width="350" src="https://i.imgur.com/bdfNqNv.png" alt="FGV EMAp" align=right />
-</div>
+| <a href=https://www.djangoproject.com/><img width="298" height="100%" alt="Django" src="https://i.imgur.com/Z9wo3bS.png"></a> | <a href=https://postgis.net/documentation/><img width="298" height="100%" alt="postgis" src="https://i.imgur.com/pVEX2Gl.png">|<a href=https://docs.docker.com/get-started/><img width="298" height="100%" alt="docker" src="https://i.ibb.co/Yp8B38R/docker.png"> |
+|:-------------------------:|:-------------------------:|:-------------------------:|
+|<a href=https://docs.celeryq.dev/en/stable/><img width="298" height="100%" alt="celery" src="https://i.ibb.co/L81p2zD/celery.png"> | <a href=https://docs.nginx.com/><img width="298" height="100%" alt="nginx" src="https://i.ibb.co/2n5HZBg/nginx.png">|<a href=https://plotly.com/><img width="298" height="100%" alt="plotly" src="https://i.ibb.co/r0HYsYH/plotly.png">|
 
 ---
-      
-      
diff --git a/docker/celery/run.sh b/containers/celery/run.sh
similarity index 100%
rename from docker/celery/run.sh
rename to containers/celery/run.sh
diff --git a/docker/compose-base.yaml b/containers/compose-base.yaml
similarity index 80%
rename from docker/compose-base.yaml
rename to containers/compose-base.yaml
index 24505f15..34152761 100644
--- a/docker/compose-base.yaml
+++ b/containers/compose-base.yaml
@@ -30,7 +30,7 @@ services:
   base:
     build:
       context: ".."
-      dockerfile: "docker/web/Dockerfile-base"
+      dockerfile: "containers/web/Dockerfile-base"
       args:
         HOST_UID: ${HOST_UID}
         HOST_GID: ${HOST_GID}
@@ -47,13 +47,13 @@ services:
       - CELERY_BROKER_URL=amqp://guest:guest@rabbitmq:5672
       - PSQL_HOST=${PSQL_HOST}
       - PSQL_PORT=${PSQL_PORT}
-    # volumes:
-    #   - ${DOCKER_HOST_MEDIA_ROOT}:/MEDIA_ROOT
-    #   - ${DOCKER_HOST_IMPORTED_FILES_DIR}:/IMPORTED_FILES
-    #   - ${DOCKER_HOST_SHAPEFILES_DIR}:/srv/deploy/shapefile
-    #   - ${DOCKER_HOST_INCIDENCE_MAPS}:/opt/services/AlertaDengue/static/img/incidence_maps
-    #   - ${DOCKER_HOST_TEMP_PARQUET_DIR}:/tmp/dbfs_parquet
-    #   - ${DOCKER_HOST_STATIC}:/opt/services/staticfiles
+    volumes:
+      - ${DOCKER_HOST_MEDIA_ROOT}:/MEDIA_ROOT
+      - ${DOCKER_HOST_IMPORTED_FILES_DIR}:/IMPORTED_FILES
+      - ${DOCKER_HOST_SHAPEFILES_DIR}:/srv/deploy/shapefile
+      - ${DOCKER_HOST_INCIDENCE_MAPS}:/opt/services/AlertaDengue/static/img/incidence_maps
+      - ${DOCKER_HOST_TEMP_PARQUET_DIR}:/tmp/dbfs_parquet
+      - ${DOCKER_HOST_STATIC}:/opt/services/staticfiles
     depends_on:
       - worker
       - rabbitmq
@@ -61,7 +61,7 @@ services:
     restart: always
     healthcheck:
       test: ["CMD", "curl", "-f", "http://localhost:8000"]
-      interval: 20s
+      interval: 10s
       timeout: 10s
       retries: 5
       start_period: 10s  # requires docker-compose 3.4
@@ -85,7 +85,7 @@ services:
     restart: always
     healthcheck:
       test: ["CMD-SHELL", 'celery --app ad_main.celeryapp.app inspect ping -d "celery@$${HOSTNAME}"']
-      interval: 60s
+      interval: 10s
       timeout: 10s
       retries: 5
       start_period: 30s
diff --git a/docker/compose-dev.yaml b/containers/compose-dev.yaml
similarity index 63%
rename from docker/compose-dev.yaml
rename to containers/compose-dev.yaml
index 8e994484..f1f15817 100644
--- a/docker/compose-dev.yaml
+++ b/containers/compose-dev.yaml
@@ -2,26 +2,32 @@ version: "3.4"
 services:
   memcached:
     networks:
-      - staging
+      - default
 
   rabbitmq:
     networks:
-      - staging
+      - default
 
   db:
     build:
       context: ".."
-      dockerfile: "docker/postgres/Dockerfile"
+      dockerfile: "containers/postgres/Dockerfile"
     env_file:
       - ../.env
+    environment:
+      POSTGRES_PASSWORD: ${PSQL_PASSWORD}
+      POSTGRES_HOST_AUTH_METHOD: "md5"
     hostname: db
+    networks:
+      - default
     ports:
       - ${PSQL_PORT}:${PSQL_PORT}
-    networks:
-      - staging
+    expose:
+      - ${PSQL_PORT}
     volumes:
-      - pg_data:/var/lib/postgresql/data
-      - ./postgres/dumps:/dumps
+      - postgres_data:/var/lib/postgresql/data/
+      - ./postgres/dev_dumps:/dumps
+    command: -p ${PSQL_PORT} # postgres -c max_wal_size=3GB
     restart: always
     healthcheck:
       test: ["CMD-SHELL", "pg_isready -p ${PSQL_PORT} -U postgres"]
@@ -38,20 +44,19 @@ services:
     ports:
       - 8080:8000
     networks:
-      - staging
+      - default
     command: ["python", "manage.py", "runserver"]
 
   worker:
     networks:
-      - staging
+      - default
     depends_on:
       - rabbitmq
     restart: always
 
+volumes:
+  postgres_data:
+
 networks:
-  staging:
+  default:
       driver: bridge
-
-
-volumes:
-  pg_data:
diff --git a/docker/compose-prod.yaml b/containers/compose-prod.yaml
similarity index 85%
rename from docker/compose-prod.yaml
rename to containers/compose-prod.yaml
index 0558bf59..be38e8a3 100644
--- a/docker/compose-prod.yaml
+++ b/containers/compose-prod.yaml
@@ -4,7 +4,7 @@ services:
   web:
     build:
       context: ".."
-      dockerfile: "docker/web/Dockerfile-prod"
+      dockerfile: "containers/web/Dockerfile-prod"
       args:
         IMAGE_BUILD: infodengue-prod_base:latest
     env_file:
diff --git a/containers/postgres/Dockerfile b/containers/postgres/Dockerfile
new file mode 100644
index 00000000..012a358f
--- /dev/null
+++ b/containers/postgres/Dockerfile
@@ -0,0 +1,19 @@
+# from https://github.com/postgis/docker-postgis/blob/969ef8a9e4a22e1556ca99cab4178f6e60c1483d/14-3.2/Dockerfile
+FROM postgres:14-bullseye
+
+LABEL maintainer="Sandro Loch <es.loch@gmail.com>"
+
+ENV PG_MAJOR 14
+ENV POSTGIS_MAJOR 3
+
+RUN apt-get update \
+      && apt-cache showpkg postgresql-$PG_MAJOR-postgis-$POSTGIS_MAJOR \
+      && apt-get install -y --no-install-recommends \
+           postgresql-$PG_MAJOR-postgis-$POSTGIS_MAJOR \
+           postgresql-$PG_MAJOR-postgis-$POSTGIS_MAJOR-scripts \
+      && rm -rf /var/lib/apt/lists/*
+
+COPY containers/postgres/postgres_setup/* /docker-entrypoint-initdb.d/
+# COPY containers/postgres/schemas/* /docker-entrypoint-initdb.d/
+
+USER postgres
diff --git a/containers/postgres/dev_dumps/.gitattributes b/containers/postgres/dev_dumps/.gitattributes
new file mode 100644
index 00000000..e69de29b
diff --git a/docker/postgres/setup/00-infodengue_roles.sql b/containers/postgres/postgres_setup/00-infodengue_roles.sql
similarity index 70%
rename from docker/postgres/setup/00-infodengue_roles.sql
rename to containers/postgres/postgres_setup/00-infodengue_roles.sql
index de6e4100..d7cb6ee4 100644
--- a/docker/postgres/setup/00-infodengue_roles.sql
+++ b/containers/postgres/postgres_setup/00-infodengue_roles.sql
@@ -1,13 +1,16 @@
 --
 -- PostgreSQL database cluster dump
 --
+
 SET default_transaction_read_only = off;
 
 SET client_encoding = 'UTF8';
 SET standard_conforming_strings = on;
+
 --
 -- Roles
 --
+
 CREATE ROLE "Dengue";
 ALTER ROLE "Dengue" WITH NOSUPERUSER INHERIT NOCREATEROLE NOCREATEDB NOLOGIN NOREPLICATION NOBYPASSRLS VALID UNTIL 'infinity';
 CREATE ROLE administrador;
@@ -17,21 +20,14 @@ ALTER ROLE dengue WITH NOSUPERUSER INHERIT NOCREATEROLE NOCREATEDB LOGIN NOREPLI
 CREATE ROLE forecast;
 ALTER ROLE forecast WITH SUPERUSER INHERIT CREATEROLE CREATEDB LOGIN REPLICATION VALID UNTIL 'infinity';
 ALTER ROLE forecast SET search_path = forecast, public;
-CREATE ROLE dengueadmin;
-ALTER ROLE dengueadmin WITH SUPERUSER INHERIT CREATEROLE CREATEDB LOGIN NOREPLICATION NOBYPASSRLS PASSWORD NULL VALID UNTIL 'infinity';
+-- CREATE ROLE dengueadmin;
+-- ALTER ROLE dengueadmin WITH SUPERUSER INHERIT CREATEROLE CREATEDB LOGIN NOREPLICATION NOBYPASSRLS PASSWORD NULL VALID UNTIL 'infinity';
 CREATE ROLE postgres;
 ALTER ROLE postgres WITH SUPERUSER INHERIT CREATEROLE CREATEDB LOGIN REPLICATION NOBYPASSRLS;
 CREATE ROLE "Read_only";
 ALTER ROLE "Read_only" WITH NOSUPERUSER INHERIT NOCREATEROLE NOCREATEDB LOGIN NOREPLICATION NOBYPASSRLS VALID UNTIL 'infinity';
-CREATE ROLE deploy;
-ALTER ROLE deploy WITH SUPERUSER INHERIT CREATEROLE CREATEDB LOGIN NOREPLICATION NOBYPASSRLS PASSWORD NULL VALID UNTIL 'infinity';
 --
 -- Role memberships
 --
 GRANT "Dengue" TO dengue GRANTED BY administrador;
 GRANT "Dengue" TO dengueadmin GRANTED BY administrador;
---
--- PostgreSQL cluster bucardo
---
-CREATE ROLE bucardo;
-ALTER ROLE bucardo WITH SUPERUSER INHERIT CREATEROLE CREATEDB LOGIN NOREPLICATION NOBYPASSRLS PASSWORD NULL VALID UNTIL 'infinity';
diff --git a/containers/postgres/postgres_setup/01-prepare_database.sh b/containers/postgres/postgres_setup/01-prepare_database.sh
new file mode 100644
index 00000000..77faed65
--- /dev/null
+++ b/containers/postgres/postgres_setup/01-prepare_database.sh
@@ -0,0 +1,44 @@
+#!/bin/bash
+
+set -e
+
+# PostgreSQL databases
+echo "CREATE DATABASE ""$PSQL_DB"" WITH OWNER '""$PSQL_USER""';" | psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER"
+echo "CREATE DATABASE ""$PSQL_DBF"" WITH OWNER '""$PSQL_USER""';" | psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER"
+
+# PostgreSQL roles
+echo "ALTER ROLE ""$PSQL_USER"" WITH PASSWORD '""$PSQL_PASSWORD""';" | psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER"
+echo "ALTER ROLE administrador WITH PASSWORD '""$PSQL_PASSWORD""';" | psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER"
+echo "ALTER ROLE forecast WITH PASSWORD '""$PSQL_PASSWORD""';" | psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER"
+
+# PostGIS
+# ref: https://github.com/postgis/docker-postgis/blob/a6a9f021e243c0b6f908cf8ad8d0ae5460dcb1b1/14-3.2/initdb-postgis.sh
+
+# Perform all actions as $POSTGRES_USER
+export PGUSER="$POSTGRES_USER"
+
+# Create the 'template_postgis' template db
+"${psql[@]}" <<- 'EOSQL'
+CREATE DATABASE template_postgis IS_TEMPLATE true;
+EOSQL
+
+# Load PostGIS into both template_database and $POSTGRES_DB
+for DB in template_postgis "$PSQL_DBF"; do
+	echo "Loading PostGIS extensions into $DB"
+	"${psql[@]}" --dbname="$DB" <<-'EOSQL'
+		CREATE EXTENSION IF NOT EXISTS postgis;
+		CREATE EXTENSION IF NOT EXISTS postgis_topology;
+		CREATE EXTENSION IF NOT EXISTS postgis_raster WITH SCHEMA public;
+EOSQL
+
+	if [ "$DB" = "$PSQL_DBF" ]; then
+		echo "Adding PostGIS attributes to $DB"
+		"${psql[@]}" --dbname="$PSQL_DBF" <<-'EOSQL'
+			GRANT ALL ON TABLE public.geometry_columns TO PUBLIC;
+			GRANT ALL ON TABLE public.spatial_ref_sys TO PUBLIC;
+			GRANT ALL ON TABLE public.raster_columns TO PUBLIC;
+			GRANT ALL ON TABLE public.raster_overviews TO PUBLIC;
+		EOSQL
+	fi
+
+done
diff --git a/containers/postgres/postgres_setup/02-dump_into_database.sh b/containers/postgres/postgres_setup/02-dump_into_database.sh
new file mode 100755
index 00000000..7b77eab6
--- /dev/null
+++ b/containers/postgres/postgres_setup/02-dump_into_database.sh
@@ -0,0 +1,5 @@
+#!/bin/bash
+set -e
+
+gunzip -c /dumps/latest_infodengue.sql.gz | psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname="$PSQL_DBF"
+gunzip -c /dumps/latest_dengue.sql.gz | psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname="$PSQL_DB"
diff --git a/docker/postgres/schema/00-roles-development.sql b/containers/postgres/schemas/00-roles-development.sql
similarity index 84%
rename from docker/postgres/schema/00-roles-development.sql
rename to containers/postgres/schemas/00-roles-development.sql
index 29da2aa5..28a8cafd 100755
--- a/docker/postgres/schema/00-roles-development.sql
+++ b/containers/postgres/schemas/00-roles-development.sql
@@ -1,8 +1,8 @@
 --
 -- Development roles
 --
--- CREATE ROLE postgres;
--- ALTER ROLE postgres WITH SUPERUSER INHERIT CREATEROLE CREATEDB LOGIN REPLICATION NOBYPASSRLS;
+CREATE ROLE postgres;
+ALTER ROLE postgres WITH SUPERUSER INHERIT CREATEROLE CREATEDB LOGIN REPLICATION NOBYPASSRLS;
 CREATE ROLE dengueuser LOGIN PASSWORD 'denguepassword';
 ALTER ROLE dengueuser WITH SUPERUSER INHERIT CREATEROLE CREATEDB LOGIN REPLICATION VALID UNTIL 'infinity';
 CREATE ROLE forecastuser;
diff --git a/docker/postgres/schema/01-schema-dengue.sql b/containers/postgres/schemas/01-schema-dengue.sql
similarity index 100%
rename from docker/postgres/schema/01-schema-dengue.sql
rename to containers/postgres/schemas/01-schema-dengue.sql
diff --git a/docker/postgres/sql_history/167-alter_table_regional_saude.sql b/containers/postgres/sql_history/167-alter_table_regional_saude.sql
similarity index 100%
rename from docker/postgres/sql_history/167-alter_table_regional_saude.sql
rename to containers/postgres/sql_history/167-alter_table_regional_saude.sql
diff --git a/docker/postgres/sql_history/167-episem.sql b/containers/postgres/sql_history/167-episem.sql
similarity index 100%
rename from docker/postgres/sql_history/167-episem.sql
rename to containers/postgres/sql_history/167-episem.sql
diff --git a/docker/postgres/sql_history/167-update_regional_saude.sql b/containers/postgres/sql_history/167-update_regional_saude.sql
similarity index 100%
rename from docker/postgres/sql_history/167-update_regional_saude.sql
rename to containers/postgres/sql_history/167-update_regional_saude.sql
diff --git a/docker/postgres/sql_history/191-1-episem2date.sql b/containers/postgres/sql_history/191-1-episem2date.sql
similarity index 100%
rename from docker/postgres/sql_history/191-1-episem2date.sql
rename to containers/postgres/sql_history/191-1-episem2date.sql
diff --git a/docker/postgres/sql_history/create_hist_uf_diseases_view.sql b/containers/postgres/sql_history/create_hist_uf_diseases_view.sql
similarity index 100%
rename from docker/postgres/sql_history/create_hist_uf_diseases_view.sql
rename to containers/postgres/sql_history/create_hist_uf_diseases_view.sql
diff --git a/docker/postgres/sql_history/create_historico_casos.sql b/containers/postgres/sql_history/create_historico_casos.sql
similarity index 100%
rename from docker/postgres/sql_history/create_historico_casos.sql
rename to containers/postgres/sql_history/create_historico_casos.sql
diff --git a/docker/postgres/sql_history/create_table_alerta_mrj_zika.sql b/containers/postgres/sql_history/create_table_alerta_mrj_zika.sql
similarity index 100%
rename from docker/postgres/sql_history/create_table_alerta_mrj_zika.sql
rename to containers/postgres/sql_history/create_table_alerta_mrj_zika.sql
diff --git a/docker/postgres/sql_history/create_table_historico_alerta_zika.sql b/containers/postgres/sql_history/create_table_historico_alerta_zika.sql
similarity index 100%
rename from docker/postgres/sql_history/create_table_historico_alerta_zika.sql
rename to containers/postgres/sql_history/create_table_historico_alerta_zika.sql
diff --git a/docker/postgres/sql_history/create_uf_total_chik_view.sql b/containers/postgres/sql_history/create_uf_total_chik_view.sql
similarity index 100%
rename from docker/postgres/sql_history/create_uf_total_chik_view.sql
rename to containers/postgres/sql_history/create_uf_total_chik_view.sql
diff --git a/docker/postgres/sql_history/create_uf_total_zika_view.sql b/containers/postgres/sql_history/create_uf_total_zika_view.sql
similarity index 100%
rename from docker/postgres/sql_history/create_uf_total_zika_view.sql
rename to containers/postgres/sql_history/create_uf_total_zika_view.sql
diff --git a/docker/postgres/sql_history/insert_probable_case_into_hist_alerta.sql b/containers/postgres/sql_history/insert_probable_case_into_hist_alerta.sql
similarity index 100%
rename from docker/postgres/sql_history/insert_probable_case_into_hist_alerta.sql
rename to containers/postgres/sql_history/insert_probable_case_into_hist_alerta.sql
diff --git a/docker/scripts/healthcheck.sh b/containers/scripts/healthcheck.sh
similarity index 100%
rename from docker/scripts/healthcheck.sh
rename to containers/scripts/healthcheck.sh
diff --git a/docker/scripts/poetry-install.sh b/containers/scripts/poetry-install.sh
similarity index 100%
rename from docker/scripts/poetry-install.sh
rename to containers/scripts/poetry-install.sh
diff --git a/docker/web/Dockerfile-base b/containers/web/Dockerfile-base
similarity index 90%
rename from docker/web/Dockerfile-base
rename to containers/web/Dockerfile-base
index b1d25f73..6b365c82 100644
--- a/docker/web/Dockerfile-base
+++ b/containers/web/Dockerfile-base
@@ -42,9 +42,9 @@ USER deploy
 COPY --chown=deploy:deploy AlertaDengue  /opt/services/AlertaDengue
 COPY --chown=deploy env.tpl pyproject.toml poetry.lock  /opt/services/
 COPY --chown=deploy:deploy conda/${ENV}.yaml /tmp/environment.yaml
-COPY --chown=deploy:deploy docker/scripts/poetry-install.sh /tmp/poetry-install.sh
-COPY --chown=deploy:deploy docker/web/entrypoint.sh /entrypoint.sh
-COPY --chown=deploy:deploy docker/celery/run.sh  /opt/services/celery-run.sh
+COPY --chown=deploy:deploy containers/scripts/poetry-install.sh /tmp/poetry-install.sh
+COPY --chown=deploy:deploy containers/web/entrypoint.sh /entrypoint.sh
+COPY --chown=deploy:deploy containers/celery/run.sh  /opt/services/celery-run.sh
 
 # Change shell to bash
 SHELL ["/bin/bash", "-c"]
diff --git a/docker/web/Dockerfile-prod b/containers/web/Dockerfile-prod
similarity index 87%
rename from docker/web/Dockerfile-prod
rename to containers/web/Dockerfile-prod
index f97971d0..9e6b5b18 100644
--- a/docker/web/Dockerfile-prod
+++ b/containers/web/Dockerfile-prod
@@ -4,7 +4,7 @@ FROM ${IMAGE_BUILD}
 USER root
 
 # Configure cron for DBF request
-COPY docker/web/cron-tasks /etc/cron.d/cron-tasks
+COPY containers/web/cron-tasks /etc/cron.d/cron-tasks
 RUN chmod 0644 /etc/cron.d/cron-tasks \
   && crontab -u deploy /etc/cron.d/cron-tasks \
   && echo "deploy"  > /etc/cron.allow \
diff --git a/docker/web/cron-tasks b/containers/web/cron-tasks
similarity index 100%
rename from docker/web/cron-tasks
rename to containers/web/cron-tasks
diff --git a/docker/web/entrypoint.sh b/containers/web/entrypoint.sh
similarity index 100%
rename from docker/web/entrypoint.sh
rename to containers/web/entrypoint.sh
diff --git a/docker/web/prepare_permission.sh b/containers/web/prepare_permission.sh
similarity index 100%
rename from docker/web/prepare_permission.sh
rename to containers/web/prepare_permission.sh
diff --git a/docker/postgres/Dockerfile b/docker/postgres/Dockerfile
deleted file mode 100644
index 8e94b5ab..00000000
--- a/docker/postgres/Dockerfile
+++ /dev/null
@@ -1,7 +0,0 @@
-FROM mdillon/postgis:9.6
-
-# import the schemas for testing
-# ADD docker/postgres/schema/* /docker-entrypoint-initdb.d/
-
-# Dump dump for development
-ADD docker/postgres/schema/* /docker-entrypoint-initdb.d/
diff --git a/docker/postgres/setup/01-alter_password.sh b/docker/postgres/setup/01-alter_password.sh
deleted file mode 100755
index 15cd0e87..00000000
--- a/docker/postgres/setup/01-alter_password.sh
+++ /dev/null
@@ -1,4 +0,0 @@
-#!/bin/bash
-set -e
-echo "ALTER ROLE administrador WITH PASSWORD '""$PSQL_PASSWORD""';" | psql -v ON_ERROR_STOP=1 --username "$PSQL_USER"
-echo "ALTER ROLE forecast WITH PASSWORD '""$PSQL_PASSWORD""';" | psql -v ON_ERROR_STOP=1 --username "$PSQL_USER"
diff --git a/docker/postgres/setup/01-create_dengue_db.sh b/docker/postgres/setup/01-create_dengue_db.sh
deleted file mode 100755
index 6bbf170e..00000000
--- a/docker/postgres/setup/01-create_dengue_db.sh
+++ /dev/null
@@ -1,7 +0,0 @@
-#!/bin/bash
-set -e
-psql -v ON_ERROR_STOP=1 --username "$PSQL_USER" <<-EOSQL
-    CREATE DATABASE dengue WITH OWNER dengueadmin ENCODING 'utf-8';
-EOSQL
-
-gunzip -c /dumps/latest_dengue.sql.gz | psql -v ON_ERROR_STOP=1 --username "$PSQL_USER" --dbname=dengue
diff --git a/docker/postgres/setup/01-create_infodengue_db.sh b/docker/postgres/setup/01-create_infodengue_db.sh
deleted file mode 100755
index bf31c4a6..00000000
--- a/docker/postgres/setup/01-create_infodengue_db.sh
+++ /dev/null
@@ -1,7 +0,0 @@
-#!/bin/bash
-set -e
-psql -v ON_ERROR_STOP=1 --username "$PSQL_USER" <<-EOSQL
-    CREATE DATABASE infodengue WITH OWNER dengueadmin ENCODING 'utf-8';
-EOSQL
-
-gunzip -c /dumps/latest_infodengue.sql.gz | psql -v ON_ERROR_STOP=1 --username "$PSQL_USER" --dbname=infodengue
diff --git a/docs/deploying_to_staging_environment.md b/docs/deploying_to_staging_environment.md
deleted file mode 100644
index 5dba8d60..00000000
--- a/docs/deploying_to_staging_environment.md
+++ /dev/null
@@ -1,16 +0,0 @@
-Deploying to a staging environment
-----------------------------------
-
-You will need a fresh dump of the development environment on
-./dev_dumps/ (in the root directory of the repository). You also need
-to set the following environment variables in your local `.env` file
-(along with all the other ones that were already set):
-
-* `PSQL_PASSWORD`
-* `PSQL_USER`
-* `PSQL_HOST=staging_db` - you need this value for `PSQL_HOST` as it will be the hostname for the database container
-* `STAGING_PG_DATA` - this is where the db container will keep the postgres data directory
-
-
-After that, all you need to do is `make deploy_staging` and
-docker-compose will build all the images and run them.


Test patch:
diff --git a/containers/postgres/dev_dumps/latest_infodengue.sql.gz b/containers/postgres/dev_dumps/latest_infodengue.sql.gz
new file mode 100644
index 00000000..84818570
Binary files /dev/null and b/containers/postgres/dev_dumps/latest_infodengue.sql.gz differ