Portal:Toolforge/Nodes

== Overview ==
The list of Toolforge nodes. This page provides information about what kind of nodes exists in the cluster, what every node does and a prospective list of canary server for common operations like testing new stuff.
= Complete list of nodes =
The source of trust for this information is at https://openstack-browser.toolforge.org/project/tools
Take this list as an example, and refer to the source of trust for actual/current data.
tools-bastion-02.tools.eqiad1.wikimedia.cloud 
tools-bastion-03.tools.eqiad1.wikimedia.cloud 
tools-bastion-05.tools.eqiad1.wikimedia.cloud 
tools-checker-01.tools.eqiad1.wikimedia.cloud 
tools-checker-02.tools.eqiad1.wikimedia.cloud 
tools-clushmaster-02.tools.eqiad1.wikimedia.cloud 
tools-cron-01.tools.eqiad1.wikimedia.cloud 
tools-docker-builder-05.tools.eqiad1.wikimedia.cloud 
tools-docker-registry-01.tools.eqiad1.wikimedia.cloud 
tools-docker-registry-02.tools.eqiad1.wikimedia.cloud 
tools-elastic-01.tools.eqiad1.wikimedia.cloud 
tools-elastic-02.tools.eqiad1.wikimedia.cloud 
tools-elastic-03.tools.eqiad1.wikimedia.cloud 
tools-exec-1401.tools.eqiad1.wikimedia.cloud 
tools-exec-1402.tools.eqiad1.wikimedia.cloud 
tools-exec-1403.tools.eqiad1.wikimedia.cloud 
tools-exec-1404.tools.eqiad1.wikimedia.cloud 
tools-exec-1405.tools.eqiad1.wikimedia.cloud 
tools-exec-1406.tools.eqiad1.wikimedia.cloud 
tools-exec-1407.tools.eqiad1.wikimedia.cloud 
tools-exec-1408.tools.eqiad1.wikimedia.cloud 
tools-exec-1409.tools.eqiad1.wikimedia.cloud 
tools-exec-1410.tools.eqiad1.wikimedia.cloud 
tools-exec-1411.tools.eqiad1.wikimedia.cloud 
tools-exec-1412.tools.eqiad1.wikimedia.cloud 
tools-exec-1413.tools.eqiad1.wikimedia.cloud 
tools-exec-1414.tools.eqiad1.wikimedia.cloud 
tools-exec-1415.tools.eqiad1.wikimedia.cloud 
tools-exec-1416.tools.eqiad1.wikimedia.cloud 
tools-exec-1417.tools.eqiad1.wikimedia.cloud 
tools-exec-1418.tools.eqiad1.wikimedia.cloud 
tools-exec-1419.tools.eqiad1.wikimedia.cloud 
tools-exec-1420.tools.eqiad1.wikimedia.cloud 
tools-exec-1421.tools.eqiad1.wikimedia.cloud 
tools-exec-1422.tools.eqiad1.wikimedia.cloud 
tools-exec-1423.tools.eqiad1.wikimedia.cloud 
tools-exec-1424.tools.eqiad1.wikimedia.cloud 
tools-exec-1425.tools.eqiad1.wikimedia.cloud 
tools-exec-1426.tools.eqiad1.wikimedia.cloud 
tools-exec-1427.tools.eqiad1.wikimedia.cloud 
tools-exec-1428.tools.eqiad1.wikimedia.cloud 
tools-exec-1429.tools.eqiad1.wikimedia.cloud 
tools-exec-1430.tools.eqiad1.wikimedia.cloud 
tools-exec-1431.tools.eqiad1.wikimedia.cloud 
tools-exec-1432.tools.eqiad1.wikimedia.cloud 
tools-exec-1433.tools.eqiad1.wikimedia.cloud 
tools-exec-1434.tools.eqiad1.wikimedia.cloud 
tools-exec-1435.tools.eqiad1.wikimedia.cloud 
tools-exec-1436.tools.eqiad1.wikimedia.cloud 
tools-exec-1437.tools.eqiad1.wikimedia.cloud 
tools-exec-1438.tools.eqiad1.wikimedia.cloud 
tools-exec-1439.tools.eqiad1.wikimedia.cloud 
tools-exec-1440.tools.eqiad1.wikimedia.cloud 
tools-exec-1441.tools.eqiad1.wikimedia.cloud 
tools-exec-1442.tools.eqiad1.wikimedia.cloud 
tools-exec-gift-trusty-01.tools.eqiad1.wikimedia.cloud 
tools-flannel-etcd-01.tools.eqiad1.wikimedia.cloud 
tools-flannel-etcd-02.tools.eqiad1.wikimedia.cloud 
tools-flannel-etcd-03.tools.eqiad1.wikimedia.cloud 
tools-grid-master.tools.eqiad1.wikimedia.cloud 
tools-grid-shadow.tools.eqiad1.wikimedia.cloud 
tools-k8s-etcd-01.tools.eqiad1.wikimedia.cloud 
tools-k8s-etcd-02.tools.eqiad1.wikimedia.cloud 
tools-k8s-etcd-03.tools.eqiad1.wikimedia.cloud 
tools-k8s-master-01.tools.eqiad1.wikimedia.cloud 
tools-logs-02.tools.eqiad1.wikimedia.cloud 
tools-mail-02.tools.eqiad1.wikimedia.cloud 
tools-package-builder-01.tools.eqiad1.wikimedia.cloud 
tools-paws-master-01.tools.eqiad1.wikimedia.cloud 
tools-paws-worker-1001.tools.eqiad1.wikimedia.cloud 
tools-paws-worker-1002.tools.eqiad1.wikimedia.cloud 
tools-paws-worker-1003.tools.eqiad1.wikimedia.cloud 
tools-paws-worker-1005.tools.eqiad1.wikimedia.cloud 
tools-paws-worker-1006.tools.eqiad1.wikimedia.cloud 
tools-paws-worker-1007.tools.eqiad1.wikimedia.cloud 
tools-paws-worker-1010.tools.eqiad1.wikimedia.cloud 
tools-paws-worker-1013.tools.eqiad1.wikimedia.cloud 
tools-paws-worker-1016.tools.eqiad1.wikimedia.cloud 
tools-paws-worker-1017.tools.eqiad1.wikimedia.cloud 
tools-paws-worker-1019.tools.eqiad1.wikimedia.cloud 
tools-prometheus-01.tools.eqiad1.wikimedia.cloud 
tools-prometheus-02.tools.eqiad1.wikimedia.cloud 
tools-proxy-01.tools.eqiad1.wikimedia.cloud 
tools-proxy-02.tools.eqiad1.wikimedia.cloud 
tools-redis-1001.tools.eqiad1.wikimedia.cloud 
tools-redis-1002.tools.eqiad1.wikimedia.cloud 
tools-services-01.tools.eqiad1.wikimedia.cloud 
tools-services-02.tools.eqiad1.wikimedia.cloud 
tools-static-10.tools.eqiad1.wikimedia.cloud 
tools-static-11.tools.eqiad1.wikimedia.cloud 
tools-webgrid-generic-1401.tools.eqiad1.wikimedia.cloud 
tools-webgrid-generic-1402.tools.eqiad1.wikimedia.cloud 
tools-webgrid-generic-1403.tools.eqiad1.wikimedia.cloud 
tools-webgrid-generic-1404.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1401.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1402.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1403.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1404.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1405.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1406.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1407.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1408.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1409.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1410.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1411.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1412.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1413.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1414.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1415.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1416.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1417.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1418.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1419.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1420.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1421.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1422.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1424.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1425.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1426.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1427.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1428.tools.eqiad1.wikimedia.cloud 
tools-worker-1001.tools.eqiad1.wikimedia.cloud 
tools-worker-1002.tools.eqiad1.wikimedia.cloud 
tools-worker-1003.tools.eqiad1.wikimedia.cloud 
tools-worker-1004.tools.eqiad1.wikimedia.cloud 
tools-worker-1005.tools.eqiad1.wikimedia.cloud 
tools-worker-1006.tools.eqiad1.wikimedia.cloud 
tools-worker-1007.tools.eqiad1.wikimedia.cloud 
tools-worker-1008.tools.eqiad1.wikimedia.cloud 
tools-worker-1009.tools.eqiad1.wikimedia.cloud 
tools-worker-1010.tools.eqiad1.wikimedia.cloud 
tools-worker-1011.tools.eqiad1.wikimedia.cloud 
tools-worker-1012.tools.eqiad1.wikimedia.cloud 
tools-worker-1013.tools.eqiad1.wikimedia.cloud 
tools-worker-1014.tools.eqiad1.wikimedia.cloud 
tools-worker-1015.tools.eqiad1.wikimedia.cloud 
tools-worker-1016.tools.eqiad1.wikimedia.cloud 
tools-worker-1017.tools.eqiad1.wikimedia.cloud 
tools-worker-1018.tools.eqiad1.wikimedia.cloud 
tools-worker-1019.tools.eqiad1.wikimedia.cloud 
tools-worker-1020.tools.eqiad1.wikimedia.cloud 
tools-worker-1021.tools.eqiad1.wikimedia.cloud 
tools-worker-1022.tools.eqiad1.wikimedia.cloud 
tools-worker-1023.tools.eqiad1.wikimedia.cloud 
tools-worker-1025.tools.eqiad1.wikimedia.cloud 
tools-worker-1026.tools.eqiad1.wikimedia.cloud 
tools-worker-1027.tools.eqiad1.wikimedia.cloud
= Purpose of nodes =
Explanation of what each type of node is.
== Bastion nodes ==
Servers meant to allow access to the clusters for users using SSH. You can submit your grid jobs from here.
They are not meant to run any actual workload. There are usually like 3 of them.
Nodes are like: tools-bastion-12.tools.eqiad1.wikimedia.cloud
== Kubernetes nodes ==
Nodes that run Kubernetes and hold the k8s workloads (pods).
There should be like 10 or 20 of them.
* Nodes are like: tools-k8s-worker-27.tools.eqiad1.wikimedia.cloud
* Control plane nodes are like: tools-k8s-control-1.tools.eqiad1.wikimedia.cloud
* etcd nodes are like: tools-k8s-etcd-8.tools.eqiad1.wikimedia.cloud
== Web nodes ==
Nodes that act as frontend web hosts for tools in Toolforge. 
There should be like 5 of them.
* General web proxy nodes: tools-proxy-04.tools.eqiad1.wikimedia.cloud
* Static content nodes: tools-static-14.tools.eqiad1.wikimedia.cloud
== Docker registry and harbor ==
These nodes are meant distribute docker containers.
There should be like 3 or them.
* Docker registry nodes are like: tools-docker-registry-01.tools.eqiad1.wikimedia.cloud
* Harbor nodes are like: tools-harbor-01.tools.eqiad1.wikimedia.cloud
== Elasticsearch nodes ==
Elasticsearch cluster for use by tools. All indices are available for read-only access inside Toolforge. Writing requires a username and password.
There should be 3 nodes.
* Nodes are like: tools-elastic-1.tools.eqiad1.wikimedia.cloud
== checker nodes ==
* Checker nodes are like: tools-checker-03.tools.eqiad1.wikimedia.cloud
== misc nodes ==
Some misc nodes.
* Tools puppetmaster: tools-puppetmaster-02.tools.eqiad1.wikimedia.cloud
* Cumin, control of the whole cluster using Cumin: tools-cumin-1.tools.eqiad1.wikimedia.cloud, mostly replaced by cloudcumin1001.eqaid.wmnet.
* SMTP email for the cluster: tools-mail.tools.eqiad1.wikimedia.cloud
* DEB package builder: tools-package-builder-01.tools.eqiad1.wikimedia.cloud
* Prometheus deployment nodes: tools-prometheus-03.tools.eqiad1.wikimedia.cloud
* Redis nodes: tools-redis-1001.tools.eqiad1.wikimedia.cloud
* Misc services nodes (aptly and others): tools-services-01.tools.eqiad1.wikimedia.cloud
= Canary nodes =
A pre-elected list of one node of each type you can use to test stuff before deploying changes to the whole cluster.
tools-bastion-02.tools.eqiad1.wikimedia.cloud 
tools-checker-01.tools.eqiad1.wikimedia.cloud 
tools-clushmaster-01.tools.eqiad1.wikimedia.cloud 
tools-cron-01.tools.eqiad1.wikimedia.cloud 
tools-docker-builder-05.tools.eqiad1.wikimedia.cloud 
tools-docker-registry-01.tools.eqiad1.wikimedia.cloud 
tools-elastic-01.tools.eqiad1.wikimedia.cloud 
tools-exec-1401.tools.eqiad1.wikimedia.cloud 
tools-flannel-etcd-01.tools.eqiad1.wikimedia.cloud 
tools-grid-master.tools.eqiad1.wikimedia.cloud 
tools-k8s-etcd-01.tools.eqiad1.wikimedia.cloud 
tools-k8s-master-01.tools.eqiad1.wikimedia.cloud 
tools-logs-02.tools.eqiad1.wikimedia.cloud 
tools-mail-02.tools.eqiad1.wikimedia.cloud 
tools-package-builder-01.tools.eqiad1.wikimedia.cloud 
tools-paws-master-01.tools.eqiad1.wikimedia.cloud 
tools-paws-worker-1001.tools.eqiad1.wikimedia.cloud 
tools-prometheus-01.tools.eqiad1.wikimedia.cloud 
tools-proxy-01.tools.eqiad1.wikimedia.cloud 
tools-redis-1001.tools.eqiad1.wikimedia.cloud 
tools-services-01.tools.eqiad1.wikimedia.cloud 
tools-static-10.tools.eqiad1.wikimedia.cloud 
tools-webgrid-generic-1401.tools.eqiad1.wikimedia.cloud 
tools-webgrid-lighttpd-1401.tools.eqiad1.wikimedia.cloud 
tools-worker-1001.tools.eqiad1.wikimedia.cloud 
Depending on the test or task you are doing, you may need to craft a different list:
* Different OS (Ubuntu trusty, Debian Jessie, Debian stretch)
* By Linux kernel version
* By different workload, usage or general load
* Attending to other criteria
In Toolforge clushmaster, there should be a list ready to use with clush: toolforge_canary_list.txt
It can be used like this:
user@tools-clushmaster-01:~$ clush --hostfile /etc/clustershell/toolforge_canary_list.txt "command"
[...]
== See also ==
* General Toolforge admin docs 
* Creating a new exec node
Category:Documentation