Manager Handbook for Distributed AntDB-T - P21
Published: 2023-08-24 02:53:57

Dual center disaster recovery

When the primary center is not available, the secondary center will fully take over as the new primary center. There are two ways to upgrade the backup center to the primary center: zone failover and zone switchover, which are used in different scenarios. But before executing the following commands, you need to shut down the doctor on mgr. The commands are as follows:

```
stop doctor；
```
zone failover：When the primary center node is down and cannot be recovered, which makes the primary center unavailable, execute this command, then the secondary center will be upgraded to the primary center, and the nodes in the original primary center will not continue to serve.

zone switchover：It is used to switch the primary and secondary centers temporarily, that is, the primary center is downgraded to the backup center, and the backup center is upgraded to the primary center, and all nodes are still running in the original cluster.

Standby ADBMgr to master
 

```
 mgr_ctl promote -D /home/shan2/smgr 
```
Master-standby center failover

Executed on the standby mgr:

```
zone failover zone2; 
```
The nodes of the original master center will stay in the cluster as slave after failover is complete, but the status is uninitialized and not in the cluster.

The new master center is as follows, and the corresponding nodes have been upgraded to master:

```
postgres=# list node zone zone2;  
 name  |  host  |  type  | mastername | port  | sync_state |  path  | initialized | incluster | readonly | zone   
----------+------------+--------------------+------------+-------+------------+-----------------------------------+-------------+-----------+----------+-------  
 gszone2  | localhost2 | gtmcoord master  |  |  6775 |  | /home/shan2/pgdata_xc/sgc  | t  | t  | f  | zone2  
 sc0zone2 | localhost2 | coordinator master |  |  4338 |  | /home/shan2/pgdata_xc/coord/s0  | t  | t  | f  | zone2  
 sc1zone2 | localhost1 | coordinator master |  |  4338 |  | /home/shan2/pgdata_xc/coord/s1  | t  | t  | f  | zone2  
 ds1zone2 | localhost4 | datanode master  |  | 24335 |  | /home/shan2/pgdata_xc/datanode/s1 | t  | t  | f  | zone2  
 ds0zone2 | localhost3 | datanode master  |  | 14335 |  | /home/shan2/pgdata_xc/datanode/s0 | t  | t  | f  | zone2  
(5 rows)  
```
Master-standby center switchover

Executed on the standby mgr (which has currently been upgraded to master). Note that to execute zone switchover, you must first shut down the doctor, i.e., execute the stop doctor command, otherwise there is a possibility that the switchover will fail. When the gtmcoord or coordinator has active connections, the execution of zone switchover may fail, and the switchover can be forced by adding the force parameter. The details are as follows:

```
zone switchover zone2; If there is an active connection, wait for 10s by default and wait for the connection to end, if there is an active connection after 10s, the switch fails;  
zone switchover zone2 30; If there is an active connection, wait for 30s to wait for the end of the active connection, if there is an active connection after 30s, the switchover fails;  
zone switchover zone2 force； If there is an active connection, wait for 10s by default, wait for the connection to end, if there is an active connection after 10s, force switching;  
zone zone switchover force 30; If there is an active connection, wait for 30s, wait for the connection to end, if after 30s, there is still an active connection, force the switch;  
```