vania-janet/multiturn-rag-retrieval-data huggingface dataset:
MySQL database migration using Python 

The MySQL database migration script allows you to migrate MySQL databases from one server to another. This is applicable to any platform:



* IBM Cloud® classic infrastructure to IBM Cloud VPC
* On-premises to IBM Cloud VPC
* Other cloud service providers to IBM Cloud VPC





 Prerequisites 

Review the following prerequisites before you begin your migration:



1. Set up an IBM Cloud Object Storage bucket.
2. Make sure that you have write access to the Object Storage bucket.
3. Make sure that you have connection to the source and target server from your system.
4. Ensure that Python3 (version 3.0) and Pip3 are installed on your system.







 Migration overview diagram 

Zoom

![Migration Overview Diagram](https://cloud.ibm.com/docs-content/v1/content/8aa0853d34cf2349f1d2db8e5c61e6422a580169/cloud-infrastructure/images/mysql_migration_script_python.svg)

Figure 1. Migration overview diagram





 Clone and run the database migration script 

Complete the following steps to clone and run the database migration script:



1. Run the following commands to clone the public GitHub repository:

git clone https://github.com/IBM-Cloud/vpc-migration-tools.git

cd vpc-migration-tools/db-migration/mysql
2. Install the prerequisites modules for Python by running the following commands:

pip3 install -U pip setuptools

pip3 install -r requirements.txt

setuptools facilitate packaging Python projects by enhancing the Python standard library distutils. The requirements.txt files install the Python libraries that are required in the script.
3. Run the following database migration script:

python3 db_migration.py







 Provide migration details 

After you run the database migration script, you need to provide the details for the following parameters:



* Object Storage
* Source server
set-commit-status script 

The script set status checks for commits, either for Pull (or Merge) Requests or already merged commits in a main branch.

The set-commit-status script is provided by the pipeline. It does not need to be installed. The script has the following dependencies:



*  bash
*  libstdc++ shared library
*  libgcc shared library



Make sure that the dependencies are installed in the base image that uses this tool for setting Git commit statuses.



  Usage 

The script set-commit-status requires the following parameters:



*  --repository
The repository URL. Make sure that the repository is added to the toolchain as an integration.
*  --commit-sha
The full SHA hash of the commit to set the status on
*  --state
State of the check. The state can be one of the following values: failure, pending, success, or error
*  --description Short description of the status check



The following parameters are optional:



*  --context
Context of the check, default is devsecops/<CUSTOM STAGE NAME>
For example: devsecops/unit-tests
*  --task-name
Current pipeline task name, default is "<CUSTOM STAGE NAME>" For example: unit-tests
*  --step-name
Current pipeline step name, default is run-stage



Use the following command to get help:

set-commit-status --help



  Example usage 

set-commit-status   --repository "https://url-to.application-repo.git"   --commit-sha "49756b0c7e6e89516caa76ebbc697c2d55852fbc"   --state "success"   --description "Unit tests finished running."   --context "tekton/code-unit-tests"   --task-name "code-unit-tests"   --step-name "run-stage"
Subnet Gateway 01 <br><br> * <br><br><br> 01 
 VPC 01 01 02 
 Subnets 04 04 08 
 Bastion VSI 01 01 02 
 App VSI 05 05 10 
 web VSI 05 05 10 
 Database VSI 
 Floating IP 01 01 02 



The scripts use variables to specify your information, OS, Auto scale, and Cloud Object Storage:



* API and ssh key
* OS type and image
* Local machine OS
* Database OS, image, and bandwidth
* Minimum and maximum virtual server instances that are configured for the instance group and the CPU threshold for adding virtual server instances to the pool
* Global load balancer domain name and region code
* Cloud Object Storage plan and region information



The scripts also install these software packages:



* for web vsi, Wordpress, Apache and PHP
* for app vsi, Apache and PHP
* for db vsi, Mariadb







 Script Files 

The configuration and script files are provided on the [vpc-ha-iac GitHub repository](https://github.com/IBM-Cloud/vpc-ha-iac).

To create a resilient highly available architecture, you use the files in the cross-multi-region folder.

For 3-tier highly available resilient architecture in multiple regions on virtual private cloud, modify the:



* example.userinput.auto.tfvars file and follow the instruction in the file to add your API key, region, and other variables.
* Modules (instances, load_balancers, security_groups, subnets) variables.tf or .tf file to further customize specific resource parameters. For example, security group has a boiler plate for security policies that might not encompass all the rules that are needed by your application. In that case, you need to modify it and add the appropriate rules. For more information, see Procedure Step 6.



All of the other configuration files are provided and do not need to be modified.

The IBM Cloud Provider plug-in for Terraform on IBM Cloud uses these configuration files to provision a VPC in your IBM Cloud account.

The scripts take 20 - 30 min to create the VPC and components.
Build file requirements 

If the app must be built before it can be deployed, you must include a build file in your repo. If a build script file is detected in the root directory of the repo, an automated build of the code is triggered before deployment.

Supported builders include:



* [!Ant:](http://ant.apache.org/manual/using.html)build.xml, which builds output to the ./output/ folder
* [Gradle:](https://docs.gradle.org/current/userguide/getting_started.html)/build.gradle, which builds output to the . folder
* [Grunt:](http://gruntjs.com/getting-startedthe-gruntfile)/Gruntfile.js, which builds output to the . folder
* [Maven:](http://maven.apache.org/guides/introduction/introduction-to-the-pom.html)/pom.xml, which builds output to the ./target/ folder







 Pipeline file requirements 

To configure the pipeline for the toolchain in a .bluemix directory, include a pipeline.yml file. For each pipeline.yml file in that directory, a pipeline is created when the toolchain is instantiated.

To create a pipeline file, consult the example file in the [Creating custom toolchain templates](https://github.com/open-toolchain/sdk/wiki/Creating-Custom-Toolchain-Templates) documentation. Just as when you define a pipeline in the web interface, you define a pipeline in text by creating stages and jobs, setting inputs and environment variables, and adding scripts. You can also see a number of more complex pipeline files in [this demonstration project](https://github.com/open-toolchain/toolchain-demo/tree/master/.bluemix).





 Container Dockerfile requirements
: None

Command options:

--host HOST
: Optional: The IBM Cloud Kubernetes Service API endpoint to use.

--insecure
: Allow an insecure HTTP connection.

-p
: Your IBM Cloud password.

-u
: Your IBM Cloud username.

-q
: Optional: Do not show the message of the day or update reminders.

Examples:



* Example to target the US South regional endpoint:

ibmcloud ks init --host https://us-south.containers.cloud.ibm.com
* Example to target the global endpoint again:

ibmcloud ks init --host https://containers.cloud.ibm.com







 script commands 



 ibmcloud ks script update 

Virtual Private Cloud

Classic infrastructure

Rewrite scripts that call kubernetes-service commands. Legacy-structured commands are replaced with beta-structured commands. For a list of all changes between the legacy and beta formats, see the comparison table in [Using the beta IBM Cloud Kubernetes Service plug-in](https://cloud.ibm.com/docs/containers?topic=containers-cs_cli_changelogchangelog_beta).

Most command behavior and syntax changes in version 1.0. These changes are not compatible with earlier versions. After you update your scripts, you must continue to use version 1.0 of the plug-in within the script or the environment where the script is run. Do not change the IKS_BETA_VERSION environment variable to a different version.

ibmcloud ks script update [--in-place] FILE [FILE ...]

Minimum required permissions
: None

Command options:

--in-place
: Optional: Rewrite the source file with the updated command structure. If this option is not specified, you can see a summary of the changes to the script file in STDOUT.

FILE [FILE ...]
: The file that contains the scripts that you want to update.
If you need to rename your resources after they are created, modify the input.auto.tfvars file to change the names and run terraform plan and terraform apply again. Do not use the IBM Cloud Dashboard and user interface to modify your VPC after it is created. The Terraform scripts create a complete solution and selectively modifying resources with the user interface might cause unexpected results.

If you need to remove the SAP NetWeaver 7.X and Db2 installation, go to your project folder and run terraform destroy. The terraform destroy command does not remove the VPC in this scenario because the VPC was not created with the Terraform scripts.





 Related information 

For more information about Terraform on IBM Cloud, see [Terraform on IBM Cloud getting started tutorial](https://cloud.ibm.com/docs/ibm-cloud-provider-for-terraform?topic=ibm-cloud-provider-for-terraform-getting-started).

For more information about using Terraform for creating only a VPC for SAP, without the SAP architecture, see [Creating single-tier VPC for SAP on IBM Cloud® VPC with Terraform](https://cloud.ibm.com/docs/sap?topic=sap-create-terraform-single-tier-vpc-sap).

SAP One Support Notes that apply to this document:



* [SAP Note 84555 - Windows Server, Linux®, and UNIX: Certified hardware](https://launchpad.support.sap.com//notes/84855)
* [SAP Note 2927211 - SAP Applications on IBM Cloud Virtual Private Cloud (VPC) Infrastructure environment](https://launchpad.support.sap.com//notes/2927211)
* [SAP Note 2923773 - Linux® on IBM Cloud (IaaS): Adaption of your SAP License](https://launchpad.support.sap.com//notes/2923773)
* [SAP Note 2414097 - SAP Applications on IBM Cloud Classic Infrastructure environment](https://launchpad.support.sap.com//notes/2414097)
Enabling shell autocompletion for IBM Cloud CLI (Linux and macOS only) 

Starting from version 0.7.0, the IBM Cloud® Command Line Interface installers don't enable shell autocompletion automatically. To have the autocompletion support, you need to manually enable it. The autocompletion scripts are installed in the following locations:



*  Bash autocompletion: /usr/local/ibmcloud/autocomplete/bash_autocomplete
*  Zsh autocompletion: /usr/local/ibmcloud/autocomplete/zsh_autocomplete





  Enabling autocompletion for Linux 



*  If you're using Bash, add:

[ -f /usr/local/ibmcloud/autocomplete/bash_autocomplete ]] && source /usr/local/ibmcloud/autocomplete/bash_autocomplete into one of the following files:
*  For Login shell: /.bash_profile
*  For Non-login shell: /.bashrc
*  If you're using Zsh: add:

source /usr/local/ibmcloud/autocomplete/zsh_autocomplete into /.zshrc.







  Enabling autocompletion support for macOS 



*  If you're using Bash, add:

source /usr/local/ibmcloud/autocomplete/bash_autocomplete into /.bash_profile.
*  If you're using Zsh, add:

source /usr/local/ibmcloud/autocomplete/zsh_autocomplete into /.zshrc.
The Terraform scripts create a complete solution and selectively modifying resources with the user interface might cause unexpected results.

If you need to remove your VPC, go to your project folder and run terraform destroy.





 Related information 

For more information about Terraform on IBM Cloud, see [Terraform on IBM Cloud getting started tutorial](https://cloud.ibm.com/docs/ibm-cloud-provider-for-terraform?topic=ibm-cloud-provider-for-terraform-getting-started).

For more information about using Terraform for creating only a VPC for SAP, without the SAP architecture, see [Creating single-tier virtual private cloud for SAP by using Terraform](https://cloud.ibm.com/docs/sap?topic=sap-create-terraform-single-tier-vpc-sap).

SAP One Support Notes that apply to this document:



* [SAP Note 84555 - Windows Server, Linux®, and UNIX: Certified hardware](https://launchpad.support.sap.com//notes/84855)
* [SAP Note 2927211 - SAP Applications on IBM Cloud Virtual Private Cloud (VPC) Infrastructure environment](https://launchpad.support.sap.com//notes/2927211)
* [SAP Note 2923773 - Linux® on IBM Cloud (IaaS): Adaption of your SAP License](https://launchpad.support.sap.com//notes/2923773)
* [SAP Note 2414097 - SAP Applications on IBM Cloud Classic Infrastructure environment](https://launchpad.support.sap.com//notes/2414097)
* [SAP Note 2369910 - SAP Software on Linux®: General information](https://launchpad.support.sap.com//notes/2369910)
* [SAP Note 171380 - Released IBM hardware (Intel processors) and IBM cloud services offers](https://launchpad.support.sap.com//notes/171380)
* [SAP Note 1380654 - SAP support in IaaS environments](https://launchpad.support.sap.com//notes/1380654)
When an expired or expiring certificate must be renewed, and after the new certificate is ready, delete the existing certificate and then add the new one.

When you use a custom domain to serve the SSL certificate, use the following region endpoints to provide the URL route for your organization in IBM Cloud:



Table 1. Cloud Foundry custom domain region endpoints

 Region Endpoint 

 US-South custom-domain.us-south.cf.cloud.ibm.com 
 US-East custom-domain.us-east.cf.cloud.ibm.com 
 EU-DE custom-domain.eu-de.cf.cloud.ibm.com 
 EU-GB custom-domain.eu-gb.cf.cloud.ibm.com 
 AU-SYD custom-domain.au-syd.cf.cloud.ibm.com 



To upload a certificate for your Cloud Foundry app, complete the following steps:



1. From the [IBM Cloud console](https://cloud.ibm.com), click Manage, and select Account.
2. Click Cloud Foundry orgs.
3. From the Actions column, click the Actions icon ![More Actions icon](https://cloud.ibm.com/docs-content/v1/content/1e0945ee44f87283927af9404a8f2d077786db0c/icons/action-menu-icon.svg), and select Domains.
4. Click Upload for your custom domain.
5. Select an option, upload the file, and click Add.



* Certificate: A digital document that binds a public key to the identity of the certificate owner, which enables the certificate owner to be authenticated. A certificate is issued by a certificate authority and is digitally signed by that authority. A certificate is generally issued and signed by a certificate authority. However, for testing and development purposes, you might use a self-signed certificate.
* Private key: An algorithmic pattern that is used to encrypt messages that only the corresponding public key can decrypt. The private key is also used to decrypt messages that were encrypted by the corresponding public key.
Define an alias for your custom domain by specifying the IBM-provided subdomain as a Canonical Name record (CNAME).



5. In a web browser, enter the URL to access your app through the subdomain that you created.



Next, you can [enable health checks on the subdomain by creating a health monitor](https://cloud.ibm.com/docs/containers?topic=containers-loadbalancer_hostnameloadbalancer_hostname_monitor).





 Understanding the subdomain format 

Subdomains for NLBs follow the format <cluster_name>-<globally_unique_account_HASH>-0001.<region>.containers.appdomain.cloud.

For example, a subdomain that you create for an NLB might look like mycluster-a1b2cdef345678g9hi012j3kl4567890-0001.us-south.containers.appdomain.cloud. The following table describes each component of the subdomain.



Table 1. Understanding the NLB subdomain format

 NLB subdomain component Description 

 * The wildcard for the subdomain is registered by default for your cluster. 
 <cluster_name> The name of your cluster. - If the cluster name is 26 characters or fewer and the cluster name is unique in this region, the entire cluster name is included and is not modified: myclustername.<br><br><br><br> * If the cluster name is 26 characters or fewer and there is an existing cluster of the same name in this region, the entire cluster name is included and a dash with six random characters is added: myclustername-ABC123.<br> * If the cluster name is 26 characters or greater and the cluster name is unique in this region, only the first 24 characters of the cluster name are used: myveryverylongclusternam.<br> * If the cluster name is 26 characters or greater and there is an existing cluster of the same name in this region, only the first 17 characters of the cluster name are used and a dash with six random characters is added: myveryverylongclu-ABC123.<br><br><br>
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PORT=8080
PWD=/
SHLVL=1

The output of the call to the custom domain is mapped to the myapp application, which uses the icr.io/codeengine/helloworld sample image.





 Viewing domain mappings 



 Viewing domain mappings from the console 

You can view a listing of all automatically generated and custom domain mappings for your applications from the console. By default, the table contents are scoped to custom domain mappings. Use the Type filter to modify the view.

This view displays information about the expiration of the certificate that is associated with your mapping. When the certificate expires, the application is no longer reachable with the domain mapping, and this condition yields an SSL error. If you have a certificate that is about to expire, [update the custom domain mapping](https://cloud.ibm.com/docs/codeengine?topic=codeengine-domain-mappingsupdate-custom-domain) to use an updated certificate.

This view also displays information about the specific application that is associated with the domain mapping, and the type of the domain mapping. For mappings that are generated by Code Engine, the type can be System-public, System-private, or System-internal. For custom domain mappings that you create, the type is Custom.



1. After your project is in Active status, click the name of your project on the [Code Engine Projects page](https://cloud.ibm.com/codeengine/projects).
2. From the Overview page, click Domain mappings.
3. From the Domain mappings page, view a listing of the defined domain mappings for your existing applications. The Type indicates whether the mapping is automatically generated or if it is a custom domain mapping.







 Viewing domain mappings with the CLI
For example, if allowed_domains has a value of example.com and allow_subdomainsis set to true, then the following subdomains are allowed: foo.example.com, bar.example.com, .example.com.

Note: This field is redundant if you use the allow_any_name option.

--allow-glob-domains (bool)
: This field indicates whether to allow glob patterns, for example, ftp.example.com, in the names that are specified in the allowed_domains field.

If set to true, clients are allowed to request private certificates with names that match the glob patterns.

--allow-any-name (bool)
: This field indicates whether to allow clients to request a private certificate that matches any common name.

--enforce-hostnames (bool)
: This field indicates whether to enforce only valid hostnames for common names, DNS Subject Alternative Names, and the host section of email addresses.

--allow-ip-sans (bool)
: This field indicates whether to allow clients to request a private certificate with IP Subject Alternative Names.

--allowed-uri-sans ([]string)
: The URI Subject Alternative Names to allow for private certificates.

Values can contain glob patterns, for example spiffe://hostname/_.

The list items must match regular expression /(.?)/. The maximum length is 100 items. The minimum length is 0 items.

--allowed-other-sans ([]string)
: The custom Object Identifier (OID) or UTF8-string Subject Alternative Names (SANs) to allow for private certificates.

The format for each element in the list is the same as OpenSSL: <oid>:<type>:<value> where the current valid type is UTF8. To allow any value for an OID, use * as its value. Alternatively, specify a single * to allow any other_sans input.

The list items must match regular expression /(.?)/. The maximum length is 100 items. The minimum length is 0 items.
For example, point your CNAME to custom-domain.us-east.cf.cloud.ibm.com.

Cloud Foundry endpoints:



* US-SOUTH - custom-domain.us-south.cf.cloud.ibm.com
* US-EAST - custom-domain.us-east.cf.cloud.ibm.com
* EU-DE - custom-domain.eu-de.cf.cloud.ibm.com
* EU-GB - custom-domain.eu-gb.cf.cloud.ibm.com
* AU-SYD - custom-domain.au-syd.cf.cloud.ibm.com









 Accessing your app 

In a browser, enter the following URL to access your app, where hostname is your host name, and mydomain is your domain name:

http://hostname.mydomain





 Removing an orphaned route 

To remove an orphaned route, run the following command:

ibmcloud app route-delete <MY_DOMAIN> -n <MY_HOSTNAME> -f

In that example, MY_DOMAIN is the name of your domain, and MY_HOSTNAME is the host name of the route for your app. For more information about the ibmcloud app route-delete command, enter the command ibmcloud app route-delete -h.
Define an alias for your custom domain by specifying the IBM-provided subdomain as a Canonical Name record (CNAME).



5. In a web browser, enter the URL to access your app through the subdomain that you created.



Next, you can [enable health checks on the subdomain by creating a health monitor](https://cloud.ibm.com/docs/openshift?topic=openshift-loadbalancer_hostnameloadbalancer_hostname_monitor).





 Understanding the subdomain format 

Subdomains for NLBs follow the format <cluster_name>-<globally_unique_account_HASH>-0001.<region>.containers.appdomain.cloud.

For example, a subdomain that you create for an NLB might look like mycluster-a1b2cdef345678g9hi012j3kl4567890-0001.us-south.containers.appdomain.cloud. The following table describes each component of the subdomain.



Table 1. Understanding the NLB subdomain format

 NLB subdomain component Description 

 * The wildcard for the subdomain is registered by default for your cluster. 
 <cluster_name> The name of your cluster.
To customize routing for Ingress, you can add [Kubernetes NGINX annotations](https://cloud.ibm.com/docs/containers?topic=containers-comm-ingress-annotations) (nginx.ingress.kubernetes.io/<annotation>). Custom IBM Cloud Kubernetes Service annotations (ingress.bluemix.net/<annotation>) are not supported.

tls.hosts
: To use TLS, replace <domain> with the IBM-provided Ingress subdomain or your custom domain.

tls.secretName
: Replace <tls_secret_name> with the name of the Kubernetes secret where your [TLS certificate](https://cloud.ibm.com/docs/containers?topic=containers-managed-ingress-setupmanaged-ingress-steps-tls) is stored.

host
: Replace <domain> with the IBM-provided Ingress subdomain or your custom domain.

path
: Replace <app_path> with a slash or the path that your app is listening on. The path is appended to the specified Ingress domain to create a unique route to your app. When you enter this route into a web browser, network traffic is routed to the ALB. The ALB looks up the associated service and sends network traffic to the service. The service then forwards the traffic to the pods where the app runs.

pathType
: The URL path matching method. Supported values are ImplementationSpecific, Exact, or Prefix. For more information about and examples of each path type, see the [community Kubernetes documentation](https://kubernetes.io/docs/concepts/services-networking/ingress/path-types).

service.name
: Replace app1_service and app2_service, and so on, with the name of the services you created to expose your apps. Make sure that the apps you specify are in the same namespace.

service.name