Package: macrometa-target-mysql
macrometa-target-mongomacrometa-target-mysqlis a Macrometa connector for writing data into MySQL, can be used as a target for any
Data Mesh Integration.InstallationUse PIP for installation:pipinstallmacrometa-target-mysqlConfigurationThe available configuration options formacrometa-target-mysqlare:PropertyTypeRequired?DescriptionhostStringYesMySQL hostportIntYesMySQL portusernameStringYesMySQL userpasswordPasswordYesMySQL passworddatabaseStringYesMySQL database nametarget_tableStringYesDestination table namebatch_flush_sizeIntNoMaximum size of batch. Exceeding this will trigger a batch flushbatch_flush_intervalIntNoTime between batch flush executionshard_deleteBooleanNoWhenhard_deleteoption is true, DELETE SQL commands will be performed in MySQL to delete rowsadd_metadata_columnsBooleanNoMetadata columns add extra row level information about data ingestion,(i.e. when was the row read in source, when was inserted or deleted in MySQL etc.) Metadatacolumns are created automatically by adding extra columns to the tables with a column prefix_SDC_.The column names are following the stitch naming conventions documented atlink.sslBooleanNoIf set totruethen use SSL for connecting with MySQL.If the server does not accept SSL connections or the client certificate is not recognizedthen the connection will fail.ssl_check_hostnameBooleanNoFlag to configure whether SSL handshake should verify that the certificatematches the DB hostname.ssl_root_ca_certFileNoSpecific CA certificate in PEM string format. This is most often the casewhen usingself-signedserver certificate.ssl_client_certificateFileNoSpecific client certificate in PEM string format. The private key for the clientcertificate should be specified in a different parameter, SSL Client Key.ssl_client_keyFileNoSpecific client key in PEM string format.Configurations can be stored in a JSON configuration file and specified using the--configflag withmacrometa-target-mysql.Usagecat<input_stream>|macrometa-target-mysql--config<config.json><input_stream>: Input data stream<config.json>: JSON configuration filemacrometa-target-mysqlreads data from a Singer Tap and writes it to a MySQL database. Run Singer Tap to generate
data before launchingmacrometa-target-mysql.Here's an example of using Singer Tap withmacrometa-target-mysql:tap-exchangeratesapi|target-mysql--configconfig.jsonIn this case,tap-exchangeratesapiis a Singer Tap that generates exchange rate data. The data is passed tomacrometa-target-mysqlthrough a pipe(|), andmacrometa-target-mysqlwrites it to a MySQL database.config.jsoncontainsmacrometa-target-mysqlsettings.Developer ResourcesInitializing the Development Environmentpipxinstallpoetry
poetryinstallCreating and Running TestsCreate tests in themacrometa_target_mysql/testssubfolder and run:poetryrunpytestUsepoetry runto testmacrometa-target-mysqlCLI interface:poetryruntarget-mysql--helpTesting withMeltanoNote:This target functions within a Singer environment and does not require Meltano.Firstly, install Meltano and necessary plugins:# Install Meltanopipxinstallmeltano# Initialize Meltano in this directorycdtarget-mysql
meltanoinstallThen, test and orchestrate with Meltano:# Call tests:meltanoinvokemacrometa-target-mysql--version# Or execute pipeline with Carbon Intensity sample tap:meltanoruntap-carbon-intensitytarget-mysqlReference LinksMeltano Target SDK DocumentationSinger SpecificationMeltanoSinger.io