MySQL Change Data Capture Overview

Overview

Change Data Capture (CDC) is a technique that captures database inserts, updates and deletes (along with DDL changes) and replays it in the target data warehouse.

Datacoral's MySQL Change Data Capture (CDC) Slice reads the Row-Based-Replication log of MySQL, allows you to track data changes within MySQL and store them in a data warehouse. CDC can be implemented for various tasks such as auditing, copying data to another system or processing events.

The primary goal of MySQL CDC is to ensure reliable capture of data changes in MySQL and store them in a data warehouse. We utilize MySQL binary logs, which is the most efficient way to track data changes .

MySQL data changes can be captured in near-real time. Once the data is captured, it is then passed over to another process which ensures reliable storage in a data warehouse and availability for further data analysis. MySQL CDC also allows you to recover from failures such that we can pick up from when the failure happened. Our paging mechanism allows quick recovery for each failed page which results in not having to re-read the whole binary log when a failure is encountered. Datacoral TimeLabel Clock ensures that we can reprocess the exact set of changes which were lost in order to ensure accuracy of data in the warehouse.

Since a MySQL binary log contains change records from hundreds of tables, we use a complex mix of streams and fan-out processing which allows us to upload multiple tables into the warehouse in parallel. This gives you the ability to capture and store multiple schemas with hundreds of tables and millions of records immediately after you install MySQL CDC.

Supported Versions

The following flavors of MySQL are supported

Flavor of MySQLSupported versions
MySQL5.5, 5.6, 5.7, 8.0
Amazon Aurora MySQL5.6, 5.7
MySQL on Amazon RDS5.5, 5.6, 5.7, 8.0

Next Steps

Additional Information

Got a question?

Please contact Datacoral's Support Team, we'd be more than happy to answer any of your questions.