Please complete these steps before adding in the UI
Step 1: Select MySQL CDC ingest connector
- From the main menu, click on Add connector
- Select MySQL CDC Ingest from the list
- Input connector name
- Choose the destination warehouse
Step 2: Provide MySQL credentials
Fill the below mandatory fields
- Host - Hostname of the source MySQL Server
- Port - Database port
- Database - Name of the source database to connect to
- Username - Database username
- Password - Database password
If you need to connect to the mysql server through ssh tunnel, please specify the input parameters below
- SSH Tunnel Host - SSH server host name
- SSH User - SSH server user name
- Private Key - SSH server private key (We support only RSA keys)
- Passphrase - SSH server passphrase
It is recommended that the connection parameters of the replica mysql server to be used for the historical syncs as it will prevent additional overhead on the master database
- Host - Hostname of the replica MySQL Server
- Port - Database port of the replica MySQL Server
- Database - Name of the replica database to connect to
- Username - Replica database username
- Password - Replica database password
Step 3: Configure source information
- Interval : Set the frequency of data extraction for the connector
- Filtering options:
- Tables to include : Add the list of tables to include here
- Tables to exclude : Add the list of tables to exclude here
Use wildcard filters or exact names
- Perform soft deletes - Captures deletes at source in the column
Regular deletes are synced to the warehouse since those deletes are recorded in the binlogs. However, any of the below commands will not be recorded in the binlogs, hence won't be deleted. We recommend use of the regular MySQL Collect slice for databases and tables where these behaviors occur normally.
- CASCADING DELETES
- DROP and recreate table using CREATE
- Click Next
Step 4: Select the loadunits that you want to add to the connector
The list of loadunits with extraction mode and schedule is displayed.
Since this a CDC connector, the extraction mode is going to be incrementalupdate for all loadunits
Select the list of loadunits that you need to include in the connector and click Finish.
You have successfully added the connector once you landed on the below page.
Now you can enable the connector using the button at the top right hand corner of the page and data should start flowing to the warehouse incrementally.