Manager Handbook for Distributed AntDB-T - P15
Tags: database
Published: 2023-08-16 02:17:25

## Expansion process
**Add a new host**
Suppose there are currently two servers in the cluster and a new server is added through expansion.
The actual expansion scenario is mostly multiplying capacity, for example, 2 servers to 4 servers, so that the data can continue to be balanced.

```
[local]:6432 antdb@postgres=# list host; 
 name  | user  | port | protocol | agentport |     address     |    adbhome      
-------+-------+------+----------+-----------+-----------------+---------------- 
 adb01 | antdb |   22 | ssh      |      8432 | 192.168.149.131 | /opt/app/antdb 
 adb02 | antdb |   22 | ssh      |      8432 | 192.168.149.132 | /opt/app/antdb 
```
add host Adds a new server.

```
[local]:6432 antdb@postgres=# add host adb03(port=22,protocol='ssh',adbhome='/opt/app/antdb',address="192.168.149.133",agentport=8432,user='antdb'); 
ADD HOST 
```
**Note:** The new host should also be configured with the same environment variables as the other hosts.
deploy the database binaries to the new server.

```
[local]:6432 antdb@postgres=# deploy adb03; 
 hostname | status | description  
----------+--------+------------- 
 adb03    | t      | success 
```
Start the agent process of the new server.

```
[local]:6432 antdb@postgres=# start agent adb03; 
 hostname | status | description  
----------+--------+------------- 
 adb03    | t      | success 
```
At this point, the new server has been added to the cluster.
**Add datanode**
Create a datanode master on the new server.

```
[local]:6432 antdb@postgres=# add datanode master dn3_1(host='adb03', port=14332,path = '/home/antdb/data/dn3_1'); 
ADD NODE 
```
You need to modify the corresponding parameters according to the actual situation.
**Relocate data**
Expand the data in dn1_1 to the new dn3_1

```
expand datanode master dn1_1 to dn3_1; 
```
If there are multiple nodes to be expanded here, you can execute the command here several times separately, e.g.: 

```
expand datanode master dn1_1 to dn3_1; 
expand datanode master dn2_1 to dn4_1; 
```
**Activate new node**
In this step, activate all new nodes that were expanded in the previous step.

```
expand activate; 
```
At this point, the expansion work has been completed, some data has been redistributed to the new nodes, and the newly inserted table data will be distributed to the new datanode according to the slicing rules.
**Data removal**
Redundant data will be generated during the data replication and relocation process, and the redundant data is marked for removal by the following commands.

```
expand clean; 
```
At this time, the expansion has been completed, and users can check the status of the cluster through the monitor all command.
For example:
_-- Add New Host_

```
add host adb03(port=22,protocol='ssh',adbhome='/opt/app/antdb',address="192.168.149.133",agentport=8432,user='antdb'); 
add host adb04(port=22,protocol='ssh',adbhome='/opt/app/antdb',address="192.168.149.134",agentport=8432,user='antdb'); 
```
_-- deploy_

```
deploy adb03 password '123456'; 
deploy adb04 password '123456'; 
```
_-- add节点_

```
add datanode master dn3_1(host='adb03', port=14332,path = '/home/antdb/data/dn3_1'); 
add datanode master dn3_1(host='adb04', port=14332,path = '/home/antdb/data/dn4_1'); 
```
_-- expand_

```
expand datanode master dn1_1 to dn3_1; 
expand datanode master dn2_1 to dn4_1; 
```
_-- activate_

```
expand activate;
```
_-- clean_

```
expand clean;
```
**_Handle expansion exceptions_**
When the user fails to expand the capacity due to improper operation, you can execute the command recover according to the prompt.
**expand datanode exception**
•basebackup failed
•Solution: Execute "expand recover basebackup fail src to dst", if the command fails, manually delete the data directory and re-execute the expand datanode command.
•Failed to update mgr information
•Solution: Execute the command "expand recover basebackup success src to dst".
**expand activate exception**
•Failed to promote the target datanode
•Solution: Delete the datanode directory and execute the expand command again.
•Failed to restart the datanode
•Solution: Start the datanode manually
•Failed to update pgxc_node
•Solution: Execute the "expand activate recover dopromote success dst" command