Toolhub.wikimedia.org

toolhub.wikimedia.org is a catalog of tools used by the members of the Wikimedia movement. See the Toolhub project page on meta for additional details about the application's features.
== Production deployment ==
Toolhub runs in the "eqiad" Kubernetes cluster. The service is current not active-active in the "codfw" cluster (T288685).
The and for the service configure connectivity to its dependencies:
* "m5" MariaDB cluster
* search-chi-{codfw,eqiad} Elasticsearch clusters
* mcrouter managed access to "eqiad-servers" memcached pool
* url-downloader.{codfw,eqiad}.wikimedia.org outbound HTTP proxy
* text-lb.{codfw,eqiad}.wikimedia.org for MediaWiki API access without envoy proxy + custom Host headers
=== Monitoring ===
* Grafana dashboard of Prometheus metrics
* OpenSearch dashboard of log events
=== Datacenter failover ===
Toolhub does not currently operate active-active or have a documented procedure for switching traffic away from the eqiad datacenter. There is no writable database for Toolhub in codfw nor is there currently an Elasticsearch index. During the spring 2023 datacenter switchover Toolhub was left in eqiad with an expectation that it would experience some downtime as various maintenance operations happened in the largely offline DC.
=== How to deploy ===
# Create a patch to update main_app.version in to the desired tag of the wikimedia/wikimedia-toolhub container. (example)
# CR+2 the helmfile.d patch in gerrit to start the merge process.
# Ssh to the currently active deployment server (deployment.eqiad.wmnet).
# cd /srv/deployment-charts/helmfile.d/services/toolhub
# Verify that your changes from step 1 are checked out. The git clone is updated once a minute by a timer, so if your changes are not present yet double check that zuul has merged the change and then wait patiently for the update to be fetched.
# Deploy the chart to the staging cluster: helmfile -e staging -i apply --context 5
# Deploy the chart to the codfw cluster: helmfile -e codfw -i apply --context 5
# Deploy the chart to the eqiad cluster: helmfile -e eqiad -i apply --context 5
If your deployment includes database or elasticsearch schema changes, you will need to apply them manually.
* If you happen to have root rights on the deployment server you can use kube_env admin eqiad to get credentials that will allow you to use kubectl exec -n toolhub -it <pod name> toolhub-main -- /bin/bash to attach to a running container.
* If you are not a root you will need to invent your own environment for performing these post-deploy tasks until T290357: Maintenance environment needed for running one-off commands is resolved. See T290357#7578866 for the script that uses to run a container on his laptop connected to the production database and elasticsearch instance via ssh tunnels.
Typical migration commands:
* DB_USER=toolhub_admin DB_PASSWORD=<toolhub_admin's password> poetry run ./manage.py showmigrations
* DB_USER=toolhub_admin DB_PASSWORD=<toolhub_admin's password> poetry run ./manage.py migrate
* poetry run ./manage.py search_index --rebuild
== Testing deployment ==
https://toolhub-demo.wmcloud.org in the Toolhub Cloud VPS project hosts a testing deployment that is used for beta testing and demonstrations. It is deployed using docker-compose. See in the project's git repo for more information.
=== Updating the demo build ===
== Local development ==
Local development is done with docker-compose. See in the project's git repo for more information.
== Visualizations related to Tools and Account creation ==
Tools catalog relationship diagram.svg|Current understanding of how different parts, systems, and audiences related to one another for the upcoming tools catalog. 
Create a new tool via Cloud VPS (new user).svg|The workflow for creating a new tool on Cloud VPS, with three different entry points. This diagram assumes that the user does not already have an LDAP account.
Create a new tool via Cloud VPS (already on Toolforge).svg| The workflow for creating a new tool on Cloud VPS, with three different entry points. This diagram assumes that the user already has an account from using Toolforge and so they don't need to go through the registration process all over again. 
Creating a new tool on Toolforge.svg |The workflow for creating a tool on Toolforge, including four different entry points.
Category:Services