MySQL CDC Connector UI Setup Guide

Prerequisites

Please complete these steps before adding in the UI

Step 1: Select MySQL CDC ingest connector

  • From the main menu, click on Add connector
  • Select MySQL CDC Ingest from the list

  • Input connector name
  • Choose the destination warehouse

Step 2: Provide MySQL credentials

Fill the below mandatory fields

  • Host - Hostname of the source MySQL Server
  • Port - Database port
  • Database - Name of the source database to connect to
  • Username - Database username
  • Password - Database password

If you need to connect to the mysql server through ssh tunnel, please specify the input parameters below

  • SSH Tunnel Host - SSH server host name
  • SSH User - SSH server user name
  • Private Key - SSH server private key (We support only RSA keys)
  • Passphrase - SSH server passphrase

It is recommended that the connection parameters of the replica mysql server to be used for the historical syncs as it will prevent additional overhead on the master database

  • Host - Hostname of the replica MySQL Server
  • Port - Database port of the replica MySQL Server
  • Database - Name of the replica database to connect to
  • Username - Replica database username
  • Password - Replica database password

Step 3: Configure source information

  • Interval : Set the frequency of data extraction for the connector
  • Filtering options:
    • Tables to include : Add the list of tables to include here
    • Tables to exclude : Add the list of tables to exclude here

      Use wildcard filters or exact names

  • Perform soft deletes - Captures deletes at source in the column __dc_cdc_deleted_at

    Regular deletes are synced to the warehouse since those deletes are recorded in the binlogs. However, any of the below commands will not be recorded in the binlogs, hence won't be deleted. We recommend use of the regular MySQL Collect slice for databases and tables where these behaviors occur normally.

    • TRUNCATE
    • CASCADING DELETES
    • DROP and recreate table using CREATE
    • LOAD
    • RENAME
  • Click Next

Step 4: Select the loadunits that you want to add to the connector

The list of loadunits with extraction mode and schedule is displayed.

Since this a CDC connector, the extraction mode is going to be incrementalupdate for all loadunits

Select the list of loadunits that you need to include in the connector and click Finish.

You have successfully added the connector once you landed on the below page.

Now you can enable the connector using the button at the top right hand corner of the page and data should start flowing to the warehouse incrementally.

Questions?

If you have questions or feedback, feel free to reach out at hello@datacoral.co or Request a demo