Portal:Cloud VPS/Admin/notes/keystone

This page contains some notes on keystone, specifically how we deployed keystone for Cloud VPS.
= previous and current status =
Originally we had 1 keystone per deployment.
Then, Chase merged/collapsed the keystone services in labtest/labtestn, so a single keystone database serves both deployments.
This means that the keystone database stores endpoints for both deployments, making use of the openstack concept of regions.
The reason for this merge is to allow as smooth migration from the nova-network to the neutron models.
On 2018-08-13 we merged also the main/eqiad1 keystone services (see phab:T201504).
+----------------------------------+----------+--------------+--------------+---------+-----------+-----------------------------------------------------------------------+
| 28b4f3ada7984ee9aef1a40c38b52417 | eqiad1-r | glance | image | True | admin | http://cloudcontrol1003.wikimedia.org:9292 |
| c71299aff8da47c7a03f3a2184d8b64b | eqiad1-r | glance | image | True | internal | http://cloudcontrol1003.wikimedia.org:9292 |
| f922afd9417448028ce02734e0420a0b | eqiad1-r | glance | image | True | public | http://cloudcontrol1003.wikimedia.org:9292 |
| 7c1fb33eb39f4eb19be510555cebd542 | eqiad1-r | keystone | identity | True | admin | http://cloudcontrol1003.wikimedia.org:35357/v3 |
| fb991347d1d8409c9977ef0883e0917c | eqiad1-r | keystone | identity | True | internal | http://cloudcontrol1003.wikimedia.org:5000/v3 |
| be7a84a1af114f94bd1d6cc48b374413 | eqiad1-r | keystone | identity | True | public | http://cloudcontrol1003.wikimedia.org:5000/v3 |
| 30778a2f853843a3ae49a485919910fb | eqiad1-r | neutron | network | True | admin | http://cloudcontrol1003.wikimedia.org:9696 |
| 4c1a383c90344c51978b6848c22836f9 | eqiad1-r | neutron | network | True | internal | http://cloudcontrol1003.wikimedia.org:9696 |
| 0a1eb902933c4652ad41e6450fe436ee | eqiad1-r | neutron | network | True | public | http://cloudcontrol1003.wikimedia.org:9696 |
| ce6257db56114db09ea14e55c62b73cd | eqiad1-r | nova | compute | True | admin | http://cloudcontrol1003.wikimedia.org:8774/v2.1 |
| 1fe91d5e2ea344a18870a90e3abe310c | eqiad1-r | nova | compute | True | internal | http://cloudcontrol1003.wikimedia.org:8774/v2.1 |
| 4578c49346db479ab6d5b7961af8f60a | eqiad1-r | nova | compute | True | public | http://cloudcontrol1003.wikimedia.org:8774/v2.1 |
| 03ee231bc6334fd19edb32e9c6fa7465 | eqiad | designate | dns | True | admin | http://labservices1001.wikimedia.org:9001 |
| e1d0f85948f04bca9fd8d7628703a225 | eqiad | designate | dns | True | internal | http://labservices1001.wikimedia.org:9001 |
| b2621389830a4b29adb19886b8a28176 | eqiad | designate | dns | True | public | http://labservices1001.wikimedia.org:9001 |
| 655c239fd6e94e6981358811efd01b4e | eqiad | glance | image | True | admin | http://labcontrol1001.wikimedia.org:9292 |
| 1693cb5c47114161af07338765650cac | eqiad | glance | image | True | internal | http://labcontrol1001.wikimedia.org:9292 |
| a07a647fbb67452882793377ca9eab52 | eqiad | glance | image | True | public | http://labcontrol1001.wikimedia.org:9292 |
| 989fbfc7878244dab849d1c49051f2ba | eqiad | keystone | identity | True | admin | http://cloudcontrol1003.wikimedia.org:35357/v3 |
| e0ae0c5907f646fca23b13e73f2eb725 | eqiad | keystone | identity | True | internal | http://cloudcontrol1003.wikimedia.org:5000/v3 |
| 26038b5357844ec2816bbd6f06a8ee15 | eqiad | keystone | identity | True | public | http://cloudcontrol1003.wikimedia.org:5000/v3 |
| b966114d88334679b4d32f61c5dcfb31 | eqiad | nova | compute | True | admin | http://labnet1001.eqiad.wmnet:8774/v2.1 |
| 6fcd6be1f49d4d928ffedf44e6df0086 | eqiad | nova | compute | True | internal | http://labnet1001.eqiad.wmnet:8774/v2.1 |
| 4641c47238a14c68b45e831f7ec89b24 | eqiad | nova | compute | True | public | http://labnet1001.eqiad.wmnet:8774/v2.1 |
| 45f80e2d476b4408b388152e8edc1eac | eqiad | proxy | proxy | True | admin | http://proxy-eqiad.wmflabs.org:5668/dynamicproxy-api/v1/$(tenant_id)s |
| 73b78e5e22004fb098ec7dca57b0b873 | eqiad | proxy | proxy | True | internal | http://proxy-eqiad.wmflabs.org:5668/dynamicproxy-api/v1/$(tenant_id)s |
| 32cf600161824ed5a46fd850d4bf4e27 | eqiad | proxy | proxy | True | public | http://proxy-eqiad.wmflabs.org:5668/dynamicproxy-api/v1/$(tenant_id)s |
+----------------------------------+----------+--------------+--------------+---------+-----------+-----------------------------------------------------------------------+
Please note where the databases lives:
* main/eqiad1: m5-master.eqiad.wmnet
* labtest/labtestn: labtestcontrol2001.wikimedia.org
= basic deployment =
Please follow the bootstrap instructions modules/openstack/templates/bootstrap/keystone/keystone_seed.sh.erb
= creating a shared keystone =
Things to consider when collapsing keystone to server more than one deployment (example tracking phabricator task: phab:T201504)
* networking connectivity (ferm ACLs) between the clients, the endpoints and the databases
* regions in the database are correctly set
* multi-region awareness. Lots of code using the keystone API isn't region-aware.
* nova_controller vs keystone_host: we have 2 different hiera keys for setting where the keystone daemon lives. The latter is preferred.
* endpoints are updated in the database to reflect the final environment
select * from endpoint where url LIKE "%labtestcontrol%";
UPDATE endpoint set \
 url='http://labtestcontrol2003.wikimedia.org:5000/v3' \
 WHERE url='http://labtestcontrol2001.wikimedia.org:5000/v3';
UPDATE endpoint set \
 url='http://labtestcontrol2003.wikimedia.org:35357/v3' \
 WHERE url='http://labtestcontrol2001.wikimedia.org:35357/v3';
* keystone daemon only runs where it should run (the labcontrol box in the neutron deployment)
* the new database should contain endpoints and services for both deployments being merged
* related puppet patches:
** https://gerrit.wikimedia.org/r/#/c/433734/ openstack: labtest use labtestcontrol2003 for keystone
** https://gerrit.wikimedia.org/r/#/c/437783/ openstack: labtest: keystone: delete service (collapsed)
** https://gerrit.wikimedia.org/r/#/c/436853/ openstack: allow glance to call back for token validation
** https://gerrit.wikimedia.org/r/#/c/437812/ openstack: allow designate in labtest to contact labtestn keystone