Datacoral Version 1.28.0

• Cloudformation Templates

  • Ability to see logs for batch compute
  • Ability to write data events to an SQS queue to integrate with other systems.

• Connectors

  • Pipedrive - Support for extracting labels of custom fields in deals loadunit
  • Fountain - Fix to handle non-existent records of child loadunits
  • S3 - Support for identifying non-utf8 data in S3 datasource.
  • MySQL - Allowing timelabel to be supplantable in extraction query
  • Ability to add some connectors using a more configurable flow able to configure multiple aspects of the connector including loadunits, data layouts, copy configuration. Save draft connector configurations and come back to them later.
  • Standardized loadunit tables and schema names in Snowflake

• Transformations

  • Allows for materialized views to be updated where the dependencies (FROM clause) are getting updated
  • Batch compute UDFs now showing up in the UI
  • Batch compute transformations can now be used across all input/output warehouse combinations
    • input warehouse: redshift, athena
    • output warehouse: redshift, athena
  • Batch compute transformations are shown based on the output warehouse. So, batch compute transformations writing to redshift tables will now be listed within redshift.
  • See logs from your Python batch compute UDFs in the datacoral webapp

• System improvements
  • Streamlined all of the monitoring summary views in the webapp. For all connectors, transformation and publisher, the listing of units is schedule agnostic, so that all units are on the same screen grouped by schedule. Schedule filter still works.
  • Ability to integrate into Datacoral Data Events via SQS