Historical Sync UI/CLI Guide
Prerequisites
Setup connector in UI
A full table historical sync and the status of syncs initiated by UI, CLI and auto historic sync is visible in UI.
UI Guide
Trigger historical sync
Go to the individual table and click on Trigger Historical Sync button. To initiate syncs for multiple loadunits at once please use the CLI
Check status
The status of historical sync is available in individual loadunit page and historical sync tab where there is a consolidated status of all historical syncs.
Loadunit page
Navigate to the loadunit and hover on the icon next to the Trigger Historical Sync button
The status will be updated at the bottom of the page
Connector level
The connector page will now have two tabs
Runtime - Shows the sync status of all loadunits
Historical Syncs - Shows the status of the automatic and manual historical syncs
- Loadunit Name - Displays the name of the loadunit where the the sync has begun
- Operations - Operations can be table add and column add
note
We support automatic column backfills when changes (column add/rename) are detected at source. Column rename operation is executed by deleting the column and adding it again with the new name
- Columns - Displays the name of the column within the loadunit where the schema change is detected
- Status - Can be ,
NOT-STARTED
,INPROGRESS
,SUCCESS
orFAILURE
- Timelabel - The
Timelabel
when the schema change is detected - Started At and Updated At - Marks the time of commencement and the latest status update respectively
Clicking on See Details will show the data flow
CLI Guide
Using Datacoral CLI historical syncs can be triggered. Here is the list of commands that can be used for it.
Historical sync command
Trigger historical syncs for the connector
Historical syncs for all the tables in a connector
Example
Output
Trigger historical syncs for loadunits
Multiple loadunits can be added as a comma separated string as given in the example
Example
Output
Get status for all loadunits in a connector
Example
Output
Get status for specific loadunit
Example
Output
Get detailed historical sync status of all loadunits
Example
Output
Get detailed sync status of specific loadunits
Example
Output