Repository: mydumper/mydumper

Issue #448: Need help in restoring the 700GB database using MyLoader
Created: 2021-10-23T02:29:18Z
Hi,
     I am using MyLoader to restore the 700GB database. I tried the import three time now. First I have got only one error, was able to import it without any errors second time. But however when I am running the MyLoader for third time from a similar capacity machine I am seeing multiple errors. My data dump is in Azure VM which is a 16vCore machine and I am exporting the dump into Azure managed MySQL (16vCore)

$ myloader --host= --user= --password= --directory=./backup --queries-per-transaction=500 --threads=32 --compress-protocol --ssl --verbose=3 -e 2>myloader-logs.txt
The import into Azure MySQL is taking around 10hours, I am not worried about the time taken but I want a clean run of MyLoader
I am seeing two types of errors
(myloader:27208): CRITICAL **: 03:03:44.175: Error restoring cubic.bck_c5_job_candidate_30mar19 from file cubic.bck_c5_job_candidate_30mar19.sql.gz: Lost connection to MySQL server during query
(myloader:27208): CRITICAL **: 03:03:44.176: Error switching to database cubic whilst restoring table c5_tags
Below are the current values of destination system
max_allowed_packet: 1073741824
innodb_write_io_threads: 16
Scaled up Storage tier 1500 GB
connect_timeout: 240 seconds
net_write_timeout: 6000 seconds
net_read_timeout: 120 seconds
wait_timeout: 300 seconds
I am using mydumper 0.10.1, built against MySQL 5.7.32-35
Please guide me with this issue.

Duplicate issue #1537: [HELP WANTED] myloader systematically hangs with error Lost connection to MySQL server during query
Created: 2024-06-20T12:51:08Z
I have many tables to migrate from azure mysql single server (mysql 5.7) to azure mysql flexible server (mysql 8.0).
I use mydumper/muymloader  v0.16.3-3.
Dump succeed.
But when I use myloader, I get systematically this error :
** (myloader:236887): CRITICAL **: 13:48:52.242: Error connection to database: Can't connect to MySQL server on 'az-p-xxx.mysql.database.azure.com:3306' (111)
./importagatelmollard.sh: line 4: 236887 Trace/breakpoint trap   (core dumped)
The logs ends with 2024-06-20 13:48:52 [WARNING] - Connection 1851 - ERROR 2013: Lost connection to MySQL server during query.
Command line use :
myloader --host=az-p-xxx.mysql.database.azure.com --user=xxx --password=yyy --directory=/data/$DIR --database=$DB --queries-per-transaction=500 --threads=4 --compress-protocol --verbose=3 -e -L $DB-import-logs.txt -o --max-threads-for-schema-creation=1 #--innodb-optimize-keys=AFTER_IMPORT_ALL_TABLES
I tried with innodb-optimize-keys and without.
Hope that someone cen help me...
Regards

Closing comment (2024-06-21T07:58:19Z): Hello David.

Thanks for your feedback.
I confirm that the db did not crashed as I can display data inside.

Regards

 Vincent DOYELLE
Chef de projet Infrastructure/Cloud

Tel. +33(4)79 25 33 47 / Mob +33(6)14 65 02 99
12 Avenue du Lac d'Annecy Savoie Technolac | Campus Chambery | 73081 Le Bourget du Lac
Visitez notre site internet : https://www.omneseducation.com/

De : David Ducos ***@***.***>
Envoyé : jeudi 20 juin 2024 18:30
À : mydumper/mydumper ***@***.***>
Cc : DOYELLE Vincent(A) ***@***.***>; Mention ***@***.***>
Objet : Re: [mydumper/mydumper] [HELP WANTED] myloader systematically hangs with error Lost connection to MySQL server during query (Issue #1537)


Maybe related to #448<#448>

-
Reply to this email directly, view it on GitHub<#1537 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BCWNXLN4DVHKPKIWF7XEGPLZIL7O5AVCNFSM6AAAAABJT4ZVN6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBRGA4TQMRTGI>.
You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>