Receive Datacoral Events through Slack

Internally, Datacoral services communicate with each other by sending or receiving data or metadata events. If you wish to get access to these events, you can specify a Slack endpoint which Datacoral can send data event messages to.

Step 1: Create a Slack queue

See Slack documentation on how to create a webhook.

Step 2: Go To Installation Settings

In the top-right corner of the Datacoral webapp, click on Installation Settings.

drawing

Step 3: Click on Add New Endpoint

drawing

Step 4: Configure the Slack Endpoint

Step 4.1: Add the webhook endpoint

Select Slack notification endpoint, give it a name and provide the URL for the Slack webhook you created earlier.

drawing

Step 4.2: (Optional) Specify filters

By default datacoral sends notifications for all metadata events and failure data events. If you wish to subscribe to a subset of events, you can do so by specifying the filters.

Below are the configurable filters:

  1. Event Types - data/metadata
  2. Schema Whitelist - Notifications for only the provided list of warehouse schemas will be sent.
  3. Schema Blacklist - Events for the list of warehouse schemas will not be notified.

The schema whitelist and blacklist names could also be regex expressions. Also, they work together in conjunction. For example, if we wish to receive notifications for all but one schema starting with datacoral, we can specify [ "datacoral*" ] as the Schema Whitelist and [ "datacoral_blacklist" ] as the Schema Blacklist.

Notes

  1. Slack webhook will only receive metadata events and failure data events.
  2. To receive the full set of data and metadata events, subscribe to the SQS Notifications Endpoint instead.
  3. See Handling Schema Change for details about how Metadata events are used by Datacoral to propagate schema changes at the source to the destination warehouses.

An example Slack data event notifications is below:

drawing
An example metadata event in Slack looks like the following:
drawing