Internally, Datacoral services communicate with each other by sending or receiving data or metadata events. If you wish to get access to these events, you can specify a Slack endpoint which Datacoral can send data event messages to.
Step 1: Create a Slack queue
See Slack documentation on how to create a webhook.
Step 2: Go To Installation Settings
In the top-right corner of the Datacoral webapp, click on Installation Settings.
Step 3: Click on Add New Endpoint
Step 4: Configure the Slack Endpoint
Step 4.1: Add the webhook endpoint
Select Slack notification endpoint, give it a name and provide the URL for the Slack webhook you created earlier.
Step 4.2: (Optional) Specify filters
By default datacoral sends notifications for all metadata events and failure data events. If you wish to subscribe to a subset of events, you can do so by specifying the filters.
Below are the configurable filters:
- Event Types - data/metadata
- Schema Whitelist - Notifications for only the provided list of warehouse schemas will be sent.
- Schema Blacklist - Events for the list of warehouse schemas will not be notified.
The schema whitelist and blacklist names could also be regex expressions. Also, they work together in conjunction. For example, if we wish to receive notifications for all but one schema starting with datacoral, we can specify [ "datacoral*" ] as the Schema Whitelist and [ "datacoral_blacklist" ] as the Schema Blacklist.
- Slack webhook will only receive metadata events and failure data events.
- To receive the full set of data and metadata events, subscribe to the SQS Notifications Endpoint instead.
- See Handling Schema Change for details about how Metadata events are used by Datacoral to propagate schema changes at the source to the destination warehouses.
An example Slack data event notifications is below:
An example metadata event in Slack looks like the following: