NEM-Symbol Blockchain: bootstrap to shoestring node migration and upgrade

This guide could help your NEM-Symbol blockchain node to migrate from symbol-bootstrap container (discontinued) to symbol shoestring container (python3 module container).

### DISCLAIMER
This procedure takes no responsibility on your wallet keys custodial
or any losses of XYM.
Funds key is referred hereafter as "Main key" or "Wallet key",
it should be stored safely offline. 
The procedure described here deletes "Main key" with ```rm``` command
at section "Removing plain keys for node safety".

### PREREQUISITES:
1) GNU/Linux Operating System
2) a running symbol-bootstrap blockchain node

### WHAT YOU WILL GET:
a NEM-Symbol blockchain node running on shoestring with latest catapult client version v1.0.3.8 using the same public key, the same node public key, the same remote key, the same vrf key as your existing symbol-bootstrap node.

### PYTHON AND SHOESTRING INSTALLATION
#### Install pyenv and python

```console
   curl -fsSL https://pyenv.run | bash
   echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bashrc
   echo '[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc
   echo 'eval "$(pyenv init - bash)"' >> ~/.bashrc
   echo 'alias python="$(pyenv which python)"' >> ~/.bashrc
   echo 'alias pip="$(pyenv which pip)"' >> ~/.bashrc
   pyenv install 3.11.12 
   pyenv global 3.11.12
   exec "$SHELL"
   python --version
```
for more information on this step check here 
https://github.com/pyenv/pyenv?tab=readme-ov-file#installation
https://catnotes.xyz/symbol/best-practices/python-new-project

#### Install Shoestring
```console
   pip install symbol-shoestring
   pip show symbol-shoestring
```
You should have installed Shoestring ```Version: 0.2.1```
For more information check here 
https://qiita.com/ccHarvestasya/items/1c1a60beaf3a80d912c2

 ### FIND YOUR NODE KEYS
 
 #### NEM-Symbol blockchain client keys
 To run your node you need four keys, one for each of these four types:
 - Wallet (also known as ```main``` or ```ca.key.pem```)
 - Node (also known as ```transport```)
 - Remote (the remote harvesting key is the one Wallet is linked to)
 - VRF (for Verifiable Random Function)

For more information check here
https://catnotes.xyz/symbol/transactions/understanding/links

 #### Find the keys in bootstrap
 Change directory to your symbol-boostrap node installation directory.
 Your symbol-bootstrap installation directory now on is referred here as ```.```
 where you find the ```./target``` directory and the ```./target/addresses.yml``` file.
The keys mentioned before are written in ```addresses.yml```file, immediately after these respective yml lines
```console
   main:
     privateKey: ...
     publicKey: ...
     address: ...
   transport:
     ...
   remote:
     ...
   vrf:
     ...
```

Please check with the NEM-Symbol blockchain mainnet explorer at
https://symbol.fyi that the ADDRESS of the main key is linked
with the remote public key and the vrf public key. Also check the xym balance.
You'll need to decrypt them if they are ```ENCRYPTED```, with this command
```console
symbol-bootstrap decrypt --source target/addresses.yml --destination target/d_addresses.yml
```
enter the password, now you have decrypted keys in your bootstrap node directory under ```./target/d_addresses.yml```

#### Copy the keys to shoestring node directory
 Now create a new shoestring node directory (check you don't have it already)
```mkdir -p ../shoestring-node```
copy the keys there
```console
grep -A1 main d_addresses.yml | sed -nE 's/[[:space:]]*privateKey:[[:space:]]*//p > ../shoestring-node/main_k.txt
grep -A1 transport d_addresses.yml | sed -nE 's/[[:space:]]*privateKey:[[:space:]]*//p > ../shoestring-node/transport_k.txt
grep -A1 remote d_addresses.yml | sed -nE 's/[[:space:]]*privateKey:[[:space:]]*//p > ../shoestring-node/remote_k.txt
grep -A1 vrf d_addresses.yml | sed -nE 's/[[:space:]]*privateKey:[[:space:]]*//p > ../shoestring-node/vrf_k.txt
```
#### Special case for voting keys
For voting keys we just have dat files. We copy all of them.
```console
   cp ./target/nodes/node/votingkeys/private_key*.dat ../shoestring-node
```

That's all the keys, but we need the wallet address too so execute alse
```console
grep -A3 main d_addresses.yml | sed -nE 's/[[:space:]]*address:[[:space:]]*//p > ../shoestring-node/main_a.txt
```

#### NOTE: 
The main key should not stay on your online host for security reasons.
Find your way to store offline the main key. Voting nodes have this key
being the 3M XYM wallet (could be also a multisig wallet).
The remote key is the key used for node harvesting, it is linked to your
main wallet. Your node could not harvest if this key is lost. Backup remote, and VRF accordingly.

### CREATE SHOESTRING NODE ON THE SAME KEYS
Change directory to shoestring node created before
```cd ../shoestring-node```
#### Generate and edit init file 
```
   python -m shoestring mainnet.ini --package mainnet
   vi mainnet.ini
```
Edit ```[node]``` section 
Choose ```features``` accordingly or leave it there.
Give a value to ```caCommonName``` i.e. ```caCommonName = myhost.com ``` 
Give a value to ```nodeCommonName``` i.e. ```nodeCommonName = node ```

#### Generate ovverrides.ini
```console
   printf "[node.localnode]\n" > overrides.ini
   printf "host = "$(hostname) >> overrides.ini
   printf "\nfriendlyName = friendlyname" >> overrides.ini
```
substitute friendlyname with your node friendly name at wish.

#### Generate the ca.key.pem
This the certificate contains your wallet key. Voting nodes should
have 3M xym balance in it. It is generated from ```main_k.txt```
and will be deleted from the filesystem because it should not stay
online for security reasons.
```console
   python -m shoestring pemtool --input ./main_k.txt --output ./ca.key.pem
```
 This command has generated the file ca.key.pem
 
#### Generate the node.key.pem
This certificate is the node transport and should be rotated for security reasons but in this article we want a clone of our bootstrap node so we reuse the same transport by generating it from the transport key.
```
   python -m shoestring pemtool --input ./transport_k.txt --output ./node.key.pem
```
We update the config file to instruct to use this transport key
```console
   sed -i 's%nodeKey = %& '$PWD'/node.key.pem%' mainnet.ini
```
You can check that the public key corresponds to your node in https://nemnodes.org/nodes with the value in the last column of your node's row, called node publi key, with the output of this command.
```openssl pkey -in node.key.pem -pubout -noout -text ```
 
#### Create shoestring setup
```console
   python -m shoestring setup --package mainnet --ca-key-path ./ca.key.pem --config ./mainnet.ini --overrides ./overrides.ini --directory .
```

#### Override remote harvesting key
This setup has generated a new harvesting key so we need to ovverride it using our remote key. We delete the linking transaction and we overwrite the remote key with the one we copied from bootstrap.
```console
   rm linking_transaction.dat
   python -m shoestring pemtool --input remote_k.txt --output ./keys/remote.pem
   cat remote_k.txt | xargs -I{} sed -iE 's/(harvesterSigningPrivateKey = )(.*$)/\1'{}'/' ./userconfig/resources/config-harvesting.properties
```    
#### Override VRF key
```console
   python -m shoestring pemtool --input vrf_k.txt --output ./keys/vrf.pem
   cat vrf_k.txt | xargs -I{} sed -iE 's/(harvesterVrfPrivateKey = )(.*$)/\1'{}'/' ./userconfig/resources/config-harvesting.properties
```
#### Update beneficiary address
```console
   cat main_a.txt | xargs -I{} sed -iE 's/(beneficiaryAddress =)(.*$)/\1 '{}'/' ./userconfig/resources/config-harvesting.properties
```
Please check that config-harvesting.properties file content has the correct values of our private keys 
```
harvesterSigningPrivateKey = #should have the value of the remote private key
harvesterVrfPrivateKey = #should have the value of the vrf private key
beneficiaryAddress = #should have the address of our wallet (3M for voting nodes)
```
#### Override voting keys
```
  rm -rf ./keys/voting/private_key_tree1.dat
  mv private_key_tree*.dat ./keys/voting/
```

The setup is completed.
Next step should be:
Shut down bootstrap node
Start up shoestring node from the shoestring directory
```
  docker-compose up -d
```
eventually clean docker with prune system -a

#### Removing plain keys for node safety
Remember to delete the keys once the node is functioning well
this means to delete these files after having made an offline backup
```
  target/d_addresses.yml
  shoestring_node/main_k.txt
  shoestring_node/transport_k.txt
  shoestring_node/remote_k.txt
  shoestring_node/vrf_k.txt
  shoestring_node/ca.key.pem
  shoestring_node/node.key.pem
```

### CREDITS
Greetings to dusanjp.com and his samdal network that made possible to have sperimental activity