Datacoral Version 1.17.0

  • Webapp You can now add and remove some of the collect slices from the web application. Earlier your only option for adding, removing and updating collect slices was via the CLI. The new web application experience makes it a lot easier for you by providing a point and click experience.

  • Collect Slices

    • NonDatacoral - If you already have data in a warehouse, we now provide you with the ability to use that data to create data pipelines. The NonDatacoral slice allows you to add transformations for data not ingested by Datacoral collect slices.
    • Facebook - Added ability to automatically refresh the user access token. This solves the issue where a long running backfill would error out because the token expired during backfill. We also added two more loadunits , namely - ad_accounts and ads
    • Stripe - Re-implemented the slice with support for 26 loadunits
    • Pipedrive - Fixed incremental syncs to handled delayed updates. Added fixes for deal_flows loadunit schema
    • Bug fixes for Pingdom and Hubspot slices to handle errors gracefully
  • Organize Slices

    • Automatic cleaning of manifest file on successful matview-update. Earlier, if the matview-update stage was successful, users had to manually go and clean the manifest file to create a new update set.
    • You can now pause and resume all Athena based materialized views
    • A new @description attribute has been added to the DPL file. This allows you to provide a description for your materialized views.
    • You can now create regularupsert materialized views using the dpl file.
  • Harness slices

    • Salesforce publisher
      • Added support for username/password based authentication
      • Upsert mode for publish is supported on foreign key fields