Skill: freepengyang/ops-mysql-master-slave (version 1.0.2)

--- name: ops-mysql-master-slave description: MySQL Master-Slave Setup & Troubleshooting SOP. Covers replication setup, Percona XtraBackup large-volume migration, replica lag investigation, and cross-cloud database acceleration. triggers: - "mysql master slave" - "mysql replication" - "master slave sync" - "mysql replica lag" - "mysql migration" - "percona xtrabackup" - "xtrabackup" - "database migration" - "mysql backup" category: ops tags: [mysql, replication, database, backup, percona, ops] version: 1.0.0 created: 2026-05-06 --- # MySQL Master-Slave Setup & Troubleshooting SOP ## Scenario 1: Set Up MySQL Master-Slave Replication Applicable to: Read replica setup, cross-region database acceleration, analytics query isolation. ### Prerequisites - Master and slave running the same MySQL version (recommend 5.7 or 8.0) - Network connectivity between master and slave - Sufficient disk space on slave ### Standard Steps **1. Configure master MySQL** ```bash vim /etc/my.cnf ``` Add the following: ```ini [mysqld] server-id = 1 # Must be unique log-bin = mysql-bin binlog_format = ROW ``` ```bash systemctl restart mysqld ``` **2. Configure slave MySQL** ```bash vim /etc/my.cnf ``` Add the following: ```ini [mysqld] server-id = 2 # Must be unique, different from master relay-log = relay-bin read-only = 1 # Recommended for slave ``` ```bash systemctl restart mysqld ``` **3. Create replication user on master** ```sql CREATE USER 'repl'@'%' IDENTIFIED BY 'password'; GRANT REPLICATION SLAVE ON *.* TO 'repl'@'%'; FLUSH PRIVILEGES; ``` **4. Lock tables and get binlog position** ```sql FLUSH TABLES WITH READ LOCK; SHOW MASTER STATUS; -- Record File and Position ``` **Important**: Do not close this session or stop MySQL before unlocking. **5. Export data from master** ```bash # Export all databases mysqldump -u root -p --all-databases > /tmp/all_db.sql # Export specific databases only mysqldump -u root -p --databases db1 db2 > /tmp/db.sql ``` **6. Record binlog info and unlock** ```sql SHOW MASTER STATUS; -- Record mysql-bin.000001 and Position UNLOCK TABLES; ``` **7. Import data on slave** ```bash mysql -u root -p < /tmp/all_db.sql ``` **8. Establish replication on slave** ```sql CHANGE MASTER TO MASTER_HOST = 'master_ip', MASTER_USER = 'repl', MASTER_PASSWORD = 'password', MASTER_LOG_FILE = 'mysql-bin.000001', MASTER_LOG_POS = 1234; START SLAVE; SHOW SLAVE STATUS\G; ``` **9. Verify replication** ```sql SHOW SLAVE STATUS\G; ``` Check that both `Slave_IO_Running` and `Slave_SQL_Running` are `Yes`. --- ## Scenario 2: Percona XtraBackup Large-Volume Migration Applicable to: Database sizes too large for `mysqldump` (hundreds of GB), or when downtime must be minimized. ### Why XtraBackup - Hot backup — no database downtime required - Supports parallel compression - Based on InnoDB crash recovery — guarantees data consistency ### Environment Setup **Install XtraBackup (CentOS/RHEL 7)** ```bash yum install -y https://repo.percona.com/yum/percona-release-latest.noarch.rpm yum install -y percona-xtrabackup-24 ``` For Ubuntu/Debian: ```bash apt install percona-xtrabackup-24 ``` **Database user privileges** ```sql GRANT BACKUP_ADMIN ON *.* TO 'backup_user'@'localhost'; GRANT PROCESS, RELOAD, REPLICATION CLIENT ON *.* TO 'backup_user'@'localhost'; -- Or use root if privileges are sufficient ``` ### Backup Steps **1. Full backup + compression** ```bash xtrabackup --backup \ --target-dir=/tmp/backup_full \ --compress \ --compress-threads=4 \ --user=root \ --[REDACTED_SECRET] ``` **2. Transfer to target server** ```bash scp -r /tmp/backup_full target_server:/tmp/ ``` ### Restore Steps **1. Decompress (on target server)** ```bash xtrabackup --decompress --target-dir=/tmp/backup_full ``` **2. Prepare (consistency check)** ```bash xtrabackup --prepare --target-dir=/tmp/backup_full ``` **Important**: Unprepared backups are inconsistent. Starting MySQL with unprepared files will fail. **3. Restore** > ⚠️ 执行前请确认：目标机器 IP 正确、数据库名正确、备份文件已 prepare 且可用。生产环境建议先快照或备份旧数据目录。 ```bash # Stop MySQL systemctl stop mysqld # Clear data directory rm -rf /var/lib/mysql/* # Copy back xtrabackup --copy-back --target-dir=/tmp/backup_full # Fix permissions chown -R mysql:mysql /var/lib/mysql # Start MySQL systemctl start mysqld ``` --- ## Scenario 3: Replica Lag Investigation ### Diagnostic Commands ```sql SHOW SLAVE STATUS\G; ``` Key metrics: | Field | Meaning | |-------|---------| | `Slave_IO_Running` | IO thread status | | `Slave_SQL_Running` | SQL thread status | | `Seconds_Behind_Master` | Lag in seconds (0 = no lag) | | `Read_Master_Log_Pos` | Master binlog position read | | `Relay_Log_Space` | Total relay log size | ### Common Causes | Cause | Symptom | Resolution | |-------|---------|------------| | Slave machine underpowered | High CPU/IO | Upgrade slave hardware | | Large transaction blocking | SQL thread stuck | Split large transactions | | Network latency | IO thread slow | Check master-slave network | | Replication stopped | SQL thread Stopped | `START SLAVE` to retry | ### Cross-Cloud Database Acceleration Applicable when: Remote servers need to access a central database with high latency (e.g., overseas servers accessing a domestic database). ``` Remote server → Cloud Global Accelerator → Domestic source ECS ``` **Configuration steps**: 1. Create a global acceleration instance in your cloud provider's console 2. Configure acceleration region (remote) and source IP (domestic MySQL) 3. Obtain the accelerated IP 4. Configure security group on source server to allow the accelerator's backend nodes (note: IP group may be dynamic) 5. Test: `telnet accelerated_ip 3306` --- ## Scenario 4: MySQL Memory Usage Investigation Applicable to: Self-managed MySQL with high memory usage causing performance degradation. ### Investigation Steps **1. Confirm MySQL memory usage** ```bash ps -eo pid,rss,vsz,comm | grep mysql top -b -n 1 | grep mysql ``` **2. Buffer Pool usage by database** ```sql SELECT SUBSTRING_INDEX(table_name, '/', 1) AS db, COUNT(*) * 16 / 1024 AS size_mb FROM information_schema.innodb_buffer_page_lru WHERE table_name IS NOT NULL GROUP BY db ORDER BY size_mb DESC LIMIT 20; ``` **3. Buffer Pool total usage** ```sql SELECT COUNT(*) * 16 / 1024 AS total_mb FROM information_schema.innodb_buffer_page_lru; ``` **4. MySQL memory distribution (non-buffer pool)** ```sql SELECT event_name, CURRENT_NUMBER_OF_BYTES_USED / 1024 / 1024 AS current_mb, HIGH_NUMBER_OF_BYTES_USED / 1024 / 1024 AS high_mb FROM performance_schema.memory_summary_global_by_event_name ORDER BY CURRENT_NUMBER_OF_BYTES_USED DESC LIMIT 20; ``` **5. Active connections** ```sql SHOW STATUS LIKE 'Threads_connected'; SHOW STATUS LIKE 'Max_used_connections'; ``` **6. Identify high-memory tables** ```sql SELECT table_schema AS 'db', table_name AS 'table', ROUND(data_length / 1024 / 1024, 2) AS 'data_size_mb', ROUND(index_length / 1024 / 1024, 2) AS 'index_size_mb' FROM information_schema.tables WHERE table_schema NOT IN ('mysql', 'information_schema', 'performance_schema') ORDER BY data_length DESC LIMIT 20; ``` ### Common Findings | Scenario | Indicator | Recommendation | |----------|-----------|----------------| | Single db high buffer pool | Normal, frequently accessed | Acceptable | | Buffer pool near limit | `innodb_buffer_pool_size` too large | Reduce or upgrade memory | | Non-buffer pool abnormal | Temp tables / large sorts / big queries | Optimize SQL | | Many connections | Connection pool leak or unclosed connections | Check application connection pool | --- ## Scenario 5: Cross-Server Database Migration (Comprehensive) Applicable to: Game server migration, server upgrades, regional data transfers. ### Standard Flow ``` 1. Export cross-server data ↓ 2. Upload to target machine ↓ 3. Create database and import data ↓ 4. Migrate game/service directory ↓ 5. Modify configuration files ↓ 6. Update ops/management database records ↓ 7. Batch update related service configurations ↓ 8. Restart service ``` ### Key Checkpoints | Step | Check | |------|-------| | Data export | Stop writes, lock tables or shut down service | | Data transfer | Verify md5sum | | Data import | Check error logs | | Config update | Confirm IP, port, database name | | Database update | ops_server table, management config | | Service restart | Verify process and port | --- ## Command Cheatsheet ```bash # Check replica status SHOW SLAVE STATUS\G; # Check master binlog position SHOW MASTER STATUS; # Stop/start replica STOP SLAVE; START SLAVE; # Reset replica configuration RESET SLAVE ALL; # Check connections SHOW STATUS LIKE 'Threads_connected'; # Check max connections SHOW VARIABLES LIKE 'max_connections'; # MySQL buffer pool usage by db SELECT SUBSTRING_INDEX(table_name, '/', 1) AS db, COUNT(*) * 16 / 1024 AS size_mb FROM information_schema.innodb_buffer_page_lru WHERE table_name IS NOT NULL GROUP BY db; # XtraBackup backup xtrabackup --backup --target-dir=/tmp/backup --user=root --[REDACTED_SECRET] # XtraBackup restore xtrabackup --prepare --target-dir=/tmp/backup xtrabackup --copy-back --target-dir=/tmp/backup ``` --- ## Notes 1. **server-id must be unique** — master and slave cannot have the same value 2. **Set `read-only=1` on slave** — prevents accidental writes 3. **Use XtraBackup for large volumes** — mysqldump may fail or take too long 4. **High replica lag** — check slave machine performance and network first 5. **Always confirm IP, port, database name before migration** — wrong values will break connectivity 6. **Cross-cloud migration has network latency** — consider using a cloud global acceleration service


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)