Repository: mydumper/mydumper

Issue #860: Checksum mismatches when restoring between versions
Created: 2022-10-19T21:32:20Z
Describe the Issue
I am using mydumper/loader to upgrade from mySQL 5.7 to 8.0 (our cloud hosting provider does not offer in-place upgrade).  When loading into mySQL 8.0 using checksums, I receive warnings like this:
[WARNING] - Structure checksum mismatch found for database.table. Got 'a5a324c6', expecting '2C313EF' in file: database.table-schema-checksum.
Further investigation uncovered that this is because of the difference in the way our integer columns are defined.  In the dump from 5.7, they are defined as tinyint(4), int(11), and bigint(20) - with precision/display width included.  In 8.0, the display width is deprecated and they are loaded as simply tinyint, int, and bigint.  This throws off the checksum value because the COLUMN_TYPE used in the calculation is different between versions.
To Reproduce
Commands executed:
mydumper --host hostname --user root --password pw --database dbname --outputdir dirpath/filename --ssl --compress --routines --triggers --data-checksums --schema-checksums --routine-checksums --trx-consistency-only --verbose 3 --threads 12
myloader --host hostname --user root --password pw --database dbname --directory dirpath/filename --ssl --disable-redo-log --innodb-optimize-keys --verbose 3 --threads 12 --max-threads-per-table 8 --max-threads-for-index-creation 8 --overwrite-tables
Expected behavior
This may be an edge case of how you expect this tool to be used.  However, it would be nice if the checksum validation could account for the difference in definition. (See additional context for my rudimentary workaround).
How to repeat
CREATE TABLE test_table ( id int(11) NOT NULL AUTO_INCREMENT, name varchar(255) DEFAULT NULL ) ENGINE = INNODB, AUTO_INCREMENT = 1, CHARACTER SET latin1, COLLATE latin1_swedish_ci;
Dump and load that table using --schema-checksums.  Upon load, warning will be thrown and checksum will not match, leading the user to believe that perhaps something went wrong during the load.
Environment:

OS version: [Ubuntu 20.04.5]
MyDumper version: [0.12.7-3]

Additional context
I was able to get around this by modifying the checksum query you posted here: (#361) to account for the difference in column definition syntax:
SELECT table_name, COALESCE(LOWER(CONV(BIT_XOR(CAST(CRC32(CONCAT_WS(COLUMN_NAME, ORDINAL_POSITION, DATA_TYPE, REPLACE(REPLACE(REPLACE(COLUMN_TYPE, 'int(11)', 'int'), 'bigint(20)', 'bigint'), 'tinyint(4)', 'tinyint'))) AS UNSIGNED)), 10, 16)), 0) AS crc FROM information_schema.COLUMNS WHERE TABLE_SCHEMA = 'schema_name' GROUP BY table_name ORDER BY table_name;
I ran this in source and target databases and manually validated that the checksums matched for each table when the COLUMN_TYPE value was standardized.  They did.

Duplicate issue #862: [BUG] Schema index checksum mismatch found!
Created: 2022-10-24T11:01:03Z
Hello!
I am using myloader and mydumper for taking bakup and restore them.But  when I restore backup  I encounter with errors below:
** (myloader:2868268): WARNING **: 10:04:25.349: Schema index checksum mismatch found for `db_name`.`orders`. Got '83671d4e', expecting 'CB0C64DE' in file: db_name.orders-schema-indexes-checksum
** (myloader:2868268): WARNING **: 10:04:24.369: Schema index checksum mismatch found for `db_name`.`demo__orders`. Got '40fd4431', expecting '160AD7C6' in file: db_name.demo__orders-schema-indexes-checksum
My question is that : Should I be worried? because all checksum for data and their schema was OK except these two! How does it effect? How it is happening?
mydmper command:
mydumper   --database=$DB_NAME   --user=$DB_USER   --outputdir=/Backup/$DB_NAME--$DATE   --build-empty-files   --trx-consistency-only --compress --checksum-all
myloader command:
myloader -t 32 -d db_name  --serialized-table-creation  --verbose 3 -B db_name

Closing comment (2022-11-03T12:16:22Z): Definitely this is a duplicate of #860, sorry for the back and forth...